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PREFACE 



Hiis report has been written with the intention of meeting the needs of 
persons who have differing interests in the project. For example: 

1. Persons interested in the research results of a project evaluating 
new educational media in clinical n&dicine. 

2. Persons interested in the problems involved in the developm ent of 

gate rials of sufficient quality for use in graduate Tevci 
instruction. 



Readers who are interested in this report primarily for its presentation of 
Insults will find that most of the relevant information they seek is 
in the following edd-nurabered chapters: 



Chapter One. Statement of Problem . 

Ghaptcr Ilirco."^ ^p7rimpntal Design of the Project , 

Chapter Five. Results . 

A sunmary of these chapters nay be found In the Abstract (page v). They contain 
nost of the information which properly belongs in the final report of a research 
project. Tlie other chapters and the appendices contain specific supporting data 

but also include much material which is related only indirectly to the research 
project . 



Readers of this report who are interested in the develop ment of programmed 
for use teaching a clinical discipline to undergraduate medical 
students are advised to read the following evcn*numbercd chapters: 

Chapter IVo. (Catalog of Project Materials . 

Chapter Four. Pcvglow nt of Frograword Teaching Materials. 

Chapter Six. Interpretation of Pita . — — - 

B»eso chapters contain very little quantitative data which is not also presented 
in the odd-iumbereJ chapters. Their main purpose is to present information. 
Interpretation, and at times, unsubstantiated opinion of a kind which is 
usually excluded from a research report. They document tlie history of errors, 
misconceptions, faleo starts, and snail successes in attempting to develop and 
evaluate programmed materials to teach an uncharted clinical discipline. If 
parts of these chapters sees more appropriate to a confession than to a research 
report, the authors can only express their belief that other workers in this 
field will find that the problems which these chapters raise a>v equal in value to 
the partial solutions which the other chapters present. 



ill 



The author* hope that those £ew persevering persona who are Interested In 
both the research results of the project and in the development of the materials 
which produced these results will recognise that some repetition in each chapter 
was necessary to provide continuity for the readers who uee this text as a source 
book, rather than as a work to be read as a whole. 

Finally, for all readers, there is the last chapter: 

Chapte'.' Seven. Principal Contributions of the Project . 

This is an attempt to summarise the research and the non-research findings of the 
preceding six chapters. It has the virtue of brevity. 



Augusta, Georgia 
January 1966 



ABSTRACT 



j yyjpse . _ In 1963 the Medical College of Georgia, under a grant from the U. S. 
Office of Education, undertook a study to determine whether programed Instruction 
could oe used to Improve the teaching of the management of patients with gynecologic 
neoplasms to junior medical students. Instruction In this clinical discipline 
was assumed to have a dual nature: v 

1. The teaching of a body of knowledge or "content,” much of which Is 
controversial or subject to rapid change. 

2. The teaching of the "application" of this body of knowledge to 
continually changing new contexts (patients with Individual problems). 

MATERIALS, TWo programed texts were prepared: 

1. "Content" Text. An 830-framc linearly programmed text designed to 

„ f *;? p . < ; c conventional classroom teaching of gynecologic oncology. 

2 * - A PP. l . lc *- f J. a . n A 713-frame branching text consisting of 35 case 
presentations of patients with representative pelvic tumors and 
related conditions. Ihe programming style used complex branches and 
loops, coded Information-gathering frames, and remedial referrals to 
the content" text In an attempt to simulate on paper the critical 
decision-making processes Involved In working up and caring for real 
patients. 

CRITERION MEASURES. Four special National Board Examinations In OB-GYN Neoplasm* 
were prepared Independently for this project.. The National Board Part II, Compre- 
hensive Examinations In Obstetrics and Gynecology of previous years were also used. 

Measures of the learning of "application" (patient management) were oral 
examinations conducted by a panel of visiting judges from neighboring medical 
schools, cornu ined with special tab-item tests designed to measure specific skills 
in diagnosis and management of patients with gynecologic neoplasms. 

SK SAMPLES. The junior classes (96 students each) In the School of Medicine 

In two consecutive years were each divided into matched control and experimental 
groups. 

In the second year of the project, cross-validation studies in five other 
medical schools were eonducted using similarly selected groups in controlled, 
balanced studies. 



*kd* c *l College }of Georgia i 'in 1963-64, and at five other 
1964-65, experimental students re * ‘ * 

text as a substitute for the conventional classroom 
groups. 



instruction given to the control 



EXPERlMEKiAL PLAN , 
medical schools in 



At the Medical College of Georgia in 1964-65, control students received the 
"content" twxfc and experimental students received both texts. No lectures were 
givtn. 
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ywiflft * - c PH t * nt . Testing. Ihe linear "content" text wee found in all achoola 
to be at least the equal of and usually significantly superior to conventional 
instruction in lta effectiveness in teaching gynecologic oncology, aa measured by 
tha National Board special examinations. When student* were re-teated after a 
one-year interval, no algnlf leant dlfferenco in retention was demonstrated. 

l l l* iE B^ c ^ ttot>> ■ , Ixperimental students who received the "sppllcatlons" 

text of case presentations plus ths "content" text scored higher on the tab-item 
examinations designed to measure "application" than did control students who 
received tha "content" text alone in CD thoroughness in collecting diagnostic 
™ormtton, * 2 ' * ,1,ction o£ ■PPeopriete diagnostic and therapeutic procedures, 

hi » h * r * c ? rM £ oc overall performance to the oral examinations. 
The significance level for these differences ranged from ^7l to eKO 1. Experl- 

V tU ^! nt ! not •«®‘i«e«tly different i n their selection 
of uidiii or contraindicated diagnostic information. 



&J9 to Criterion lecord s. The superior performance of experimental students 
" ' ■^ieved without an increase in their study time over that of control groups, 
and with 'a saving of faculty time equivalent to the time spent in the prepara- 
tion and presentation of tha course's conventional instruction. 



\ttltud.s Toward Texts. The reaction of nearly all students toward both 
texts was strongly positive. 
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C11APTER ONE 
STATEMENT OF PROBLEM 

I. Ba cktroir.nl 

A. p.llnlcal Clerkship*. In the final two, or "clinical" years of medical 
school, the student is expected to develop judgment and skill In the 
management of patients and in the diagnosis and treatment of their 
diseases and problems. He does this by actively participating in the 
care of patients for whom his superiors (Interns, residents, and staff 
physicians) arc directly responsible. Thfe student's commitment to patient 
care results in his absence from many of his scheduled classes and. 
conferences. In the obstetrics and gynecologic clinical clerkship at 
the Medical College of Georgia, for example, the average student in his 
junior year misses nearly a third of hia classes. This is probably of 
little consequence moot of the time, for the student masters the subject 
matter of the classes at hia own pace from such sources as standard 
textbooks. Journal articles, and his colleagues' lecture notes. In 
learning the management of patients, however, students find these sources 
Inadequate. 

®* .Peelslon-Maklog a nd_Learnlog . Students learn from active participation in 
the decia ion-making process. From this standpoint, the Important decisions 
for the student to make are those which, for good or ill, directly affect 
the welfare of the patient. He should be faced with the consequences of 
his dtclslons as they affect the patient. Under such conditions he tend* 
to be motivated,, to learn from every available source, including the library, 
his colleagues, and hit superior* in order to avoid making errors which might 



C) 



-2- 



i 

i 



have an adverse effect upon his patient. For the protection of patients in 
teaching hospitals, decision-asking of this sort is reserved for graduate 
physicians participating in internship and residency training programs of 
gradually increasing responsibility. The medical student obviously cannot 
be given responsibility for critical decisions which affect the patient's 
welfaro, and i'< therefore exposed to the decision-making process and its 
consequences for the patient only as an observer. Understandably, the con- 
sequences to which he is exposed are apt to be the favorable ones rather cnan 
the unfavorable ones. Thus, the student's exposure to tne critical processes 
of decision-making in patient management is limited both by Ills lack of 
active responsible participation and by Sais lack o£ exposure to the resuttn 
of such participation (especially if unfavorable). 

G . Traditional Approach . For the past half-ccntury the traditional method of 

increasing the student's active participation in ttie dcGisiGn-making processes 
c£ patient care has been by tutorial or "bedside" teaching, in which a tew 
students participate actively in a questlon-and-answer coverage of a particular 
patient's problem or problems, led by an experienced clinical teacher, niiis 
requires an enormous outlay of faculty time. In the Medical College of Georgia, 
as in many other schools, the ratio of students to teaching faculty is so 



large that such teaching methods, although of recognized value to the student, 
are restricted to a small role in the coverage of the curriculum. 



II. 



1953, the Medical College of Georgia, under n grant 
from the U. S. Office of Education, undertook a study to determine whether 
programmed instruction could bo used to improve the teaching of patient manage- 
ment to students in tlieir junior year ot medical school. It was recognized 




that the improvement might take any of several forms. 
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" V ‘” 8 ,h * *" ,lent ■ , ““ <“* 
More efficient use of faculty tiae. 



B * S ^- Selectlon oQub j oct Hatter , xhe subject matter sel.cted for the project 
wm gynecologic oncology: the detection, di.gno.i., and treatment of benign 
and malignant tu*>r. of the female genitalia. The .object matter seemed to 
be of appropriate length for a major reae.rch project. Conventional text- 
book. of gynecology, of the type recorded to medical atudenta, u .e from 
40,000 to 250,000 word, in preaenting thia material. The subject matter 
represents less than 5% of what the student 1. expeeted to learn during 
his junior year at the Medical College of Georgia, .nd represents Icaa than 
1% of the requirements of the four-year curriculum. The complexity and 
variety of teaching and learning problems encountered in this relatively 
small subject matter area are representative of similar problem, encountered 
in many other phases of medical teaching. 

G * a nd »AppHc«cion ." The teaching of a clinical 

discipline in medical school was assumed to have a dual nature. 

1. The teaching of a body of scientific knowledge, much of which in 

controversial or subject to rapid change. 

2. Tho teaching of the "application" of thia body of scientific 

SSS.1SE3- r 

In this project die term "content" is used to mean the body of scientific 
knowledge considered from a standpoint similar to that of conventional class- 
room lectures or chapters l„ a conventional textbook. "Content," as the term 
is used, deals with the knowledge of normal and abnormal body functions and 
structure, diseases of various organs, incidence, etiology, pathology, clinical 
course, diagnosis, treatment, prognosis, and so forth. The kind, of knowledge 
implied by the word "content" lend themselves readily to measurement by 
conventional multiple-choice testing. 
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"Application," on the other hand, ia a tera which refers to the umi of 
"content" knowledge in the diagnosis and treataent of individual patients. 

It Includes the processes of collecting appropriate inforaatlon froa the 
patient's history, froa physical exaalnatlon, froa diagnostic tests and 
procedures done in their proper sequence, correlation of this inforaatlon, 
the formulation of a working diagnosis, and the selection of appropriate 
plans for treataent or aanageaent. The knowledge and skills laplled by 
the word "application" are inappropriately measured by conventional multiple- 
choice testing. 

Pevelopasnt of Teaching Hate rials . 

Peculiarities of subject Matter . Xht application of programed instruc- 
tion to the clinical fialda of aedicine presented special probleat. A 
successful prograa would have to taka into account not only the wide 
differences of ability and preparation aaong different atdlcal students, 
but also the peculiar difficulty that for aany clinical probleaa there 
are several divergent answers, each of which if aubjected to further 
exploration alght be found acceptable. Zt aeaasd probable that straight 
linear prograaa of the type which had bean used successfully for teaching 
baaic sciences in the pre-clinical years of medical achool would lack the 
flexibility needed for teaching decision staking in areas of controversy 
of clinical Mdicineo 

2. Choice of programing atuateav. xhe first problem of thia problem was 
to develop a strategy for the prograamlng of clinical materials. linear 
programing (although non-Skinnerian) appeared to have many applications 
to ttachlng the "content" of obstetrics and gynecology. Zt seemed 
poorly ■uittd, however, to the requirement thet the student be given 
practice in the art of applying hia newly acquired ecientific knowledge 
to individual problems of patient cere. At the etert of thie project, 
it had not yet been dstarmined whet strategy or technique was beet suited 
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to this teaching probler. xt vaa anticipated that a "branching" 
foraat could be adapted to the preaentation of clinical p rob 1cm, and 
that the student could be presented with a problea case and be required 
by a series of choices and responses to diagnose and manage the patient's 
problea. The proficient student, whose responses demonstrated that he 
had mastered the problem could be advanced to new and more challenging 
case presentations. Other students, whose responses had indicated a 
deficiency, could be directed to reaedlal Material, at the completion 
of which they could return to the case presentation. 

Since this branching approach was to be an experinental ore, based on 
no oodels then available, and for wfiich no quality controls had been 
established, it saaned prudent at the outset to use linear programing 
aa wuch a a possible in presenting reaedlal or "content" Material. Xt 
waa anticipated that tha final programed text would be a "coaposite" 
of branched and linear programing techniques, utilising reaedlal loops 
and bypasses. 

3. Special gequlreasnta of texts. It was rscognired that if the prograamd 
texts to be developed fes this project were to be of aoxlaua value to 
Medical education, they would not only have to be efficient ee instru- 
ments for self-instruction; they would also have to exhibit the 
following characteristics: 

a. Student acceptability . The target population of thia project, Junior 
Medical students, are experts in learning froa conventional educational 
Media. They have proved that they can learn efficiently froa teachers, 
lecturers, texts, articles, and audio-visual Materials which ara often 
extreoely inefficient from the learner's standpoint. A | ograantd 
text, if it ware to succeed with such a population, would have to 
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gain voluntary acceptance from adults who have invested much 
time and effort in learning to make effective use of less efficient 
but more familiar educational media. 

Interspecialty transferability. If a mixture of programming techni- 
ques were to be used in order to program subject matter which was 
not suitable for programming by any one method, principles and guide- 
lines would have to be developed defining what types of subject matter 
should be programmed by which method. If programming of clinical 
material were to be of widespread value in clinical teaching, the 
principles and guidelines developed for this project would have to be 
communicable to teachers in medical specialties other than obstetrics 
and gynecology so that they might adapt the techniques of this project 
to their own specialty. 

Applicability of method to non-medical fields . Many disciplines 
wholly unrelated in "content" to those of clinical medicine require 
the student to develop problem-solving skills and behavior patterns 
which are very similar to the ones needed to solve medical problems. 
Examples of such disciplines include various engineering fields, 
business administration, social work and counseling, equipment 
maintenance and trouble shooting, criminal investigation and intelli- 
gence work, and many others. It was recognized that in the develop- 
ment of techniques to meet the teaching needs of a medical specialty, 
there was an opportunity to explore and evaluate prtgramming strategies 
which might be immediately applicable to the teaching needs of 
unrelated disciplines. 
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E * Development of Testing Procedural. 

1. Controlled study . At tJie beginning of this project, the experimental 

plan called for the comparison in a controlled experiment of the efficiency 
of learning from specially prepared programmed texts with learning 
by "traditional" methods such as lectures, lecture notes, reading 
assignments in textbooks, and other conventional media. It was hoped that 
such a study would provide answers for the many clinical teachers who 
felt that programmed instruction was fine for teaching in other areas, 
but was simply not suited to their particular teaching goals. 

2 ’ , Unc °n trolled variables. It was recognized that this study could not 
be removed from its clinical setting without destroying its significance 
for clinical teachers, yet if the study were to be conducted in its 
naturalistic setting, one would have to accept the effects of multi- 
ple uncontrolled variables. Medical students in their clinical clerkships 
learn from many important sources in addition to the formal instruction 
of the plan led curriculum. For example, they learn from patients, interns, 
residents, other students, nurses, casual conversations with doctors, etc. 
Some, but not all of these effects could be measured if not controlled. 

. 3. Crude criterion measures. As the study progressed, it became apparent 
that the effect of uncontrolled variables on student performance was of 
lest importance than the crudity of the criterion measures used to 
evaluate that performance. The accepted criterion measures for determining 
the individual competence (or incompetence) of students in clinical skills 
proved inadequate to measure the effects on student performance of 
different methods of teaching these skills. The development of special 
testing instruments to measure the effects of different methods of clinical 
teaching uas undertaken as an important part of the project shortly before 
the completion of its first 



year. 
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III. Sun ary of Problem. 

A. Froirii Development . To met t the objectives of the project. It was necessary 

1. to develop an essentially linear program In gynecologic oncology covering 
''content,” 

2. to develop a branching prograa based on a series of case presesen tat Ions 
giving the student an opportunity to practice decision making In managing 
patients with gynecologic tumors, and 

3. to combine the above materials Into a "composite" program utilizing 
case presentations, bypasses, and remedial loops to automate completely 
the classroom Instruction of gynecologic tumors and their management. 

B. Evaluation . To evaluate whether the objectives were met It was necessary 

1. to compare the "content" program, the "composite" program, and conventional 
classroom Instruction with regard to the following dependent variables: 

a. Immediate achievement of learning of "content," 

b. retention of learning of "content," and 

c. "application" of content to new contexts, and 
2 • to provide In addition 

a* study time records of all students participating In the project, 

b. data on student attitudes towards the programmed materials, and 

c. Information on methods of preparing programmed materials In a medical 
school setting. 
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CHAPTER TWO 

CATALOG OF PROJECT MATERIALS 



I* Teaching Material* . 

A ' Specific ally for the Project , For the project, 

two type, of programed text, were developed to teach medical student, the 
detection, dlagno.l. and management of benign and malignant tumor, of the 
female genitalia. 

1. Smu&jm, An 830-frame linear programed text, covering the content 
of gynecologic oncology i„ traditional didactic .equence wa. developed. 

Hila text and the lecture, which were given to control groups at the 
Medical College of Georgia presented essentially the same content. 

(See Appendix A for Sample Frames). 

2. ^ p - Hcatlon." text . A 713-fram text consisting of 35 case presentation, 
of patients with representative types of pelvic tumors and related 
condition, was prepared. The programing styla was eclectic. It made 
use of constructed responses, complex branches and loops, cod.d information- 
gathering frames, and remedial referrals to the "content” text. The 
text aimed to teach "applications" of gynecologic oncology to specific 
problems of patient care in a manner which simulated on paper the process 

of working up and caring for real patients. (See Appendix B for Sample 
Frames), 

3 * ln f;YntCOl °« lc Oncol °gy - Eight lecture, in gynecologic oncology, 

using visual aids to the maximum, were specially prepared to be of the 

highest quality possible. They were given by an experienced teacher* and 
were tape recorded. During the preparation of the teaching materials, 
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neither the lecturer nor the program writer had access to the examination 
questions of the National Board of Medical Examiners. No attempt was 
made to "teach the tests." The lecture program was used only during 
the first year of the project at the Medical College of Georgia. 

B. Teaching Materials Developed for the Project but Excluded from Research Plan . 
"Programmed Instruction Methods for Obstetrics and Gynecology," a 108-page 
programmed text, was prepared for a post-graduate course of the American College 
of Obstetricians and Gynecologists. Tills text was designed to introduce 
clinical teachers of obstetrics and gynecology to methods of preparing 
objectives for and actually writing programmed case presentations of the type 
developed for the "applications" text. 

II. Testing Material*. 

A. Testing Materials Specially Prepared for the Project . 

Spec la 1 "cont e n t 1 1 tests . Two matched Examinations in OB-GYN 
Neoplasms of 108 questions each were prepared for the project independently 
by the National Board of Medical Examiners. The two examinations, A and 
B, comprised almost all questions in the National Board 1 * pool of 
questions in gynecologic neoplasms. All questions had been used in 
previous examinations administered nationally to candidates for medical 
licensure; therefore, performance norms for each question had already 
been established. The two examinations prepared by the National Board for 
this project were carefully matched, category for category, to provido an 
equal coverage of the subject matter using questions of comparable 
difficulty so that performance scores on the two examinations were 
essentially interchangeable. Examinations A and B were used during the 
first year of the project. For the second year of the project, the 
National Board of Medical Examiners scrambled the order of the questions 
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ln examination* A and B and named the new examination* C and 0. The 
project tkua had four interchangeable examination for pre- and post- 
testing. A more detailed deacrlptlon of the conetructlon and content 
of teata A and B la preaented in Appendix C. 

2- Structured Interview* (oral examinational to — aaure learning of 

“application.” At the atart of the project no raltten material* were 
available to the lnveatlgatore to meaeure the “application" of "content" 
to the problems of patient care, in moat medical achoola ( the traditional 
method to measure thle skill has been by oral examinations, ihe project 
waa fortunate to secure the service* of two consultants* who were teachers 
of obstetrics and gynecology at neighboring medical schools and who 
agreed to serve as judges throughout the two years of tha project. Ihe 
purpose in having outside consultants perform this part of the evaluation 
was to avoid ithe bias Inherent in having examinations done by the teachers 
(lecturer* and/or programmers) of the subject matter under investigation 
or by teachers who were familiar with the capabilities of the individual 
students or with the method by which they had bean instructed. Ihe project 
would require the outside consultants to conduct nearly 200 examinations 
in the same subject matter. It was essential that the format of the oral 
examinations permit enough flexibility and variation to avoid the danger 
that the judges would rebel and quit the project because they became 
too bored to continue with * repetitive task. Ihe format of tha 
structured interviews la presented in Appendix 0. 

3. Clinical problem-eolvlnx teste . 

a * .Unforeseen need. Written tests to measure "application" wars not 
a part of the project in its original design. In early 1964. 



D « C * 0r »f , „ J * ®* nnl8 * Associate professor of Obstetrics and Cynecology, Medical Collage 
* n ? Doct ,°, r , Mlch !! 1 N * wton * Profeaaor and Chairman, Department of * 
Obetctrlce and Gynecology, University of Mississippi School of Medicine. 



however, the developing o£ ■ branching programed text to teach 
"application" forced the recognition that general proficiency in 
"application" called for proficiencies in a variety of dletlnct 
but interrelated eld lie. In the programed text, theae ekllla 
had to be identified and taught by apeclal atrataglea and formta. 

An evaluation of the vaeulta ot the firet saeslon of oral exaa- 
i nation* in January 1964 made it apparent thet euch examination! 
vara too imprecise to neaaure certain of the proficiencies in 
"application" which could be dearly identified and taught in the 
progranmd text. In the euan^r of 1964 a eat of clinical problam- 
aolvlng exaninatlona was developed to aupplenent the progran of 
special oral examination*. 

b. format of teete. The taata make use of a new format that borrows 
freely fron the techniques and principles of Van Valkcnbnrg, Noogcr 
end Neville, and those of Rlmoldl and of HcGuiro. Tho student is 
given an opportunity to take a history, do all or part of a general 
physical examination and order diagnostic studies and procedures 
in whatever sequence ha prefers. In met caeae, he my collect 
end interpret data in nearly a hundred diffarant categorise. He 
is than aaked to define the patient's problem in detail, specifying 
the patient's diagnosis, tbs sxtent of ths disease, and tha various 
complicating and subsidiary conditions. He is asked to select from 
aa many aa 50 possibilities m sequenced plan of treatment appropriate 
to the patient's problem as he has defined them. 

c * Descripti on of teete . The nine examinations developed in this format 
Included two sets of matched pairs and one aat of "triplets," In 
each matched examination tha presenting problem was identical, but 
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th« ultimata diagnosis and Craatmant wars different. It was anti- 

cipatad that tha aatchad axaalnationa could ba adalnistarad and scorad 

interchangeably. A more detailed description of each of tha nine 

testa and aaapla teat A may be found in Appendix g. 

d * Payalopment of acorlna avatam. Trial scoring systems for tha taste 

vara developed to measure the following lntardepandant skills: 

i. Diagnostic process (comprehensiveness and appropriateness 
of the diagnostic workup). 

ii. Diagnostic product (accuracy and completeness of defining 
the patient's diseases or problems). 

Therapeutic product (appropriateness of treatment or 
disposition of patient). 

Four senior resident ph/elclans in obstetrics and gynecology served 
** * criterion group and developed a tentative scoring system for 
the nine clinical problem-solving tests. Each physician working indepen- 
dently assigned each item or option to one of 20 previously defined 
categories. The categories were later given numerical weights. 

The following is a summary of the results of testing using this 
trial scoring system: 

1 * Criterion a roup performance . Resident physicians in obstet- 
rics and gynecology took divergent paths through the test* 
but achieved uniformly higher scores. Faculty members who 
took the tests took- more divergent paths than the resldente 
did in working through the teste and received divergent 
ecoras. 

Jttntof- student performance. The junior students at the 
beginning of their clinical training in obstetrics and gyne- 
cology received scores which average less than one-fifth of 
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th« scores mads by the raaidaata. At the end of their 
clinical clarkahlpa, the junior atudenta made acorea which 
approximated the variable acorea of the faculty. 

Cloae atudy of the acorlns ayatea developed by the realdenta indi- 
cated that the realdenta had a much narrower view than did faculty 
■tabera of what conatituted appropriate patient care in each caae. 
The acorlng ayatea aeverely penalised moat devlatlona froa their 
concepta of appropriate diagnosia and treataent of each caae and 
did not diatingutah between "ncn-atandard” but appropriate therapy 
aelected by aoae faculty members and non-atandard Inappropriate 
therapy aelected by many atudenta. Much of the acorlng ayatem had 
to be dlacardcd, and the remaining parta were continually modified 
during the laat year of the project. 

B. Other Evaluati ve Instruments Prepared Specially, for the Project . 

1. Timeto cr iterion roeorda . Carda and record aheeta were prepared to 
permit atudenta to record aa caaily and aa accurately as poaaible the 
time they apent atudying gynecologic oncology. Xlio forms were uaed 
by all students in control and experimental groups in all medical 
achoola participating in the project (Sam Appendix F). 

2. Attitude surveys. A 38-qucatlon attitude scale and questionnaire was 
developed for the project and was completed by all students to whom 
the programmed texts were distributed. Students were permitted to 
return the eosrpleted questionnaires anonyawualy if they wished. (Sea 
Appendix G). 

C. Othar Tea tint Materials Weed bv the Project . 

1* National Board Part lit . The examinations in Obstetrics and Gynecology 
of the National Board of Madical Examlnera, Part IS were given. Thia 
two hour, 150-queatlon comprehensive examination In obatetrlca and 
gynacology designed for administration to senior medical students who 
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are candidates for atdleal llcenaure, was administered to all Junior 
medical student. at the Medical College of Georgia .t the end of their 

Junior year and a year Inter to the same students at the end of their 
senior year* 

2 ' --- l0n * 1 Board c,t< ‘ I or Y scores . For the project, the National Board 

of Medical Examiners provided special data Including categorical analyses 
by classes and by control and experimental groups. 

fiSher Evaluative Dat a Used by the ProWt . 

1. . P re vious academic ree-y ji. At the Medical College of Georgia, and In 
other schools participating In the project, the weighted grade-point 
averages of .11 student, participating In the project were made available 
to the Research Director to permit a control In the equality of .11 
groups. 

2 * Mld-lcal College Admission Test scores . These scores were made 

available to the Research Director for further evaluation of the equality 
of the control and experimental groups. 



CHAPTER THREE 



EXPERIMENTAL DESIGN OF THE PROJECT 



Study Samples . 

A. First Year (1963-64) . 

1* Method of ■election of groups , xho study sample consisted of the 93 stu- 
dent* in the Junior class in the School of Medicine of the Medical College 
of Georgia. They wen: divided into experimental (46) *nd control (47) 
groups. This was done on a stratified random basis. Toe classes were 
first divided into thres strata (upper third, middle third, lower third) 
based on performance in the first two years of medical school. The ' 
weighted grade-point averages of all courses taken during the first two 
yssrs of medical school were used as the basis for division. Within each 
stratum, students were assigned on a random basis to experimental and 
control groups. After the division hsd been made on the basis of weighted 
grade-point averages, a further check on tho equality of the groups 
was m**o hy comparing their Medical College Admission Test scores. 

2* Reason for_s testification. The reason for the division of toe class into 
strata was that the clinical years of medicine require different skills 
and aptitudes than do the pre-clinical years. From past experience, it 
hsd bean shown that at the Medical College of Georgia superior students 
tend to maintain their position from the first two years to the last 
two years, ss do the least productive students who remain in 
difficulty throughout all four yea.’s. In the middle third of the class, 
however, there is usually considerable cliango in student ranking from 
the pre-clinical to the clinical years. 



-17- 



3. Further division of study sample . xho Junior class, after being divided 
into equal control and experimental group* of 47 and 46 students 
respectively, was further divided into two control and two experimental 
groups. The four groups were scheduled to serve clinical clerkships 

in obstetrics and gynecology in successive quarters of nine weeks each. 
The control groups served in the first and fourth quarters and the 
experimental groups served in the second and third quarters, giving 
the project an ABB f A f format. 

4. Bias of sample favoring controls . Although the aasigniient of students 
to the different groups was made in strict accordance with the above 
described stratified random process, it became necessary for reasons 
of personal hardship and administrative convenience to make certain 
adjustments and transfers. When a student was transferred from one 
group to another, it was almost always possible to replace him with 

a student from his same stratum (thirl) of the class. There was, 
however, one important exception. All students who were in actual or 
potential Jeopardy at the end of the sophomore year were prohibited from 
serving their clerkship in obstetrics and gynecology in the fourth 
quarter of the Junior year. The reasons for this were unrelated to 
the project (they were related to the early scheduling of elective 
courses) but the effeet was to eliminate students of borderline academic 
status from the last control group of the year and distribute them 
(randomly) in the first control group and the two experimental groups* 

The result was a bias in tho selection of the study sample favoring the 
controls. 

B. fleccnd Year (1964-flfi) . A new junior class was divided into control and 
experimental groups of 45 and 47 students respectively, using the same 
procedures as wore followed tho pravious year. Again D students of borderline 
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status were not assigned to the fourth quarter and hence the bias favoring 
the last control group was continued in the second year* 

C. Cro S 3 -Validation (1964-65) . In the second year of the project, five schools 
in addition to the Medical College of Georgia participated in the project 

replicating in so far as local conditions would penult the study completed 

* 

the previous year at the Medical College of Georgia. In each of these schools, 
students were divided into control and experimental groups using methods 
comparable to but not identical with those used at the Medical College of 
Georgia. In all schools the division was checked against the students* 
previous performance, based on weighted averages, and against Medical College 
Admission Test scores. Assignment of students to control and experimental 
groups was reported to be as nearly random as possible with every effort 
being made to ensure comparability of the groups, but undoubtedly, factors 
such as special administrative needs, illness or the special personal 
problems of some students, resulted in some distortion of the pattern. 

(See Table VII) 

II, Treatments. 

A. Medical College of Georgia. First Year (1963-641 . 

1, control groups. Groups A and A*, serving clerkships in obstetrics and 
gynecology in the first and last quarters of the academic year, were 
given eight weekiy lectures in gynecologic oncology, all by an experienced 
lecturer* who had taught this subject for many years at the Medical College 
of Georgia. The lecturer was asked to cover the same material that 
was to be presented in the linear "content” programmed text. He was 
provided with an outline which specified the topics to be covered and the 
degree of detail for which the students would be held responsible. This 
same outline was followed in preparing the programmed text. The lecturer 

★Doctor William S. Boyd, Associate Clinical Professor of Obstetrics and Gynecology 
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w»« given an un Halted budget to add to hie already extensive collection 
of slides and other visual aids for use with his lectures. At his 
request, his lectures were tape recorded In order that a Monograph could 
be prepared from them later. Every effort was made to Insure that the 
lectures not only covered the assigned content but ware of the highest 
quality and Interest possible. In accordance with departmental policy, 
attendance at these lectures was not compulsory (students sometimes 
had conflicting responslblltles of higher priority In the operating 
room and In the delivery room), but a roll was taken at each class. 
Students were required to record the time they spent studying gynecologic 
oncology throughout the clerkship. 

2 * Experimental aroups. Experimental groups B and B' participated in clerk- 
ships In obstetrics and gynecology In the second and third quarters. 

They received no lectures or formsl classroom Instruction in gynecologic 
oncology. Instead, at the end of the first week of the clerkship, each 
student was given the "content" programed text and was asked to complete 
It and return It before the end of the eighth week of the clerkship. 
Students In the experimental groups, like those In the control groups, were 
encouraged to do as such outside reading In gynecologic oncology as 
they wished to, were encouraged to attend weekly pathology and 
cancer conferences, and ns a normal part of their clerkship, were assigned 
patients, some of whom had gynecologic neoplasia::. Experimental students 
were asked to keep a record of their time studying gynecologic oncology 
throughout the clerkship. 

Cogparablllty o f programmed text and loci u rea. Prior to the project, 
an agreement was reached to insure that the content of the two Mthods 
of Instruction would be as nearly alike as possible. Also, an agreement 
was reached defining the use of visual materials In teaching control and 
experimental groups. The texts of these agreements may be found In 
Appendix F. 
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5. Criterion measures, both group* , ftudtnts in both the experimental 
and control groups received one Special National Board Examination In 
01-GYN Neoplasms at the beginning of each quarter , and another as 
a post-test at the end of each quarter; the National Board Comprehensive 
Examination In Obstetrics and Gynecology 9 Part XI at the end of the 
academic year, another National Board Part II examination a year later 
at the end of the senior year, and oral examinations 9 as described below* 
B. Medical College of Georgia, gecond Year (1964-65) , By the end of the first 
year of the project 9 It was clear that the linearly programmed "content" 
text was such a satisfactory replacement for. the series of lectures that 
the lecture program could be dlscontlnued 9 and In the second year of the 
project 9 the study co^ld be devoted to comparing the effectiveness of two 
different forma of programmed texts* Furthermore, the results of using 
the programmed text to teach the "content" of gynecologic oncology of the 
Medical College of Georgia had been so satisfactory that cross-validation 
studies In other medical schools seemed warranted* Therefore, In the summer 
of 1964 the experimental plan for the school yea- of the project was 
modified as foUovss 

1* Control groups * Control groups, A and A 1 , serving clinical clerkships 
In obstetrics end gynecology In the first and fourth quarters of 
the academic year 1964*65 received In «11 respects the same treatment 
that experimental groups had received the previous year* There were no 
formal classes In gynecologic oncology; they received the linearly 
programmed "content" texts at the time of the pre-tests at the beginning 
of the clerkship and were asked to return them at the time of the post- 
tests* 

2. l&gerlmental^groups . Experimental groups, B and B 1 , had their clinical 
clerkships in obstetrics end gynecology In the second and third quarters 
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°f the academic yaar. They racalved at tha time of eha pra-taete, 

»t tha baginning of eha clarkahlp, both tha Unaarly programmed "content" 
taxt and tha branching "appllcatlona" taxt of caaa praaantatlona. Thaaa 
two taxta, uaad togathar, formed tha "compoalte" taxt apaclflad in 

tha original propoaal. Student* vara aakad to raturn tha taxta at tha 
tlae of tha poat-taata. 

3 * S ^asks. SSSSSSSx groupa . i„ ,u raapacta, tha taatlng of control 
•nd experimental groupa In tha aecond yaar of the projact waa tha earn* 

•• that In tha flrat yaar, axcapt that all atudanta racalvad additional 
tab-ltaa examination* In caaa-praaentatlon format*. Ihraa wara glvan 
•• pre-te*t* f tnd five were given ee poet-t eefce. 

C * S H i y- 8ch00l, - ( 19 64-65? « In the acadealc year 1964-65 five medical achoola 
participated In a replication of the atudy conducted at the Hedlcel College 
of Georgia the prevloua yaar. In *, C h achool, tha experimental plan waa 
Modified to Meet the requirement* of the achool** aatabllahad curriculum. 

In aplte of modification*, however, the treatment* In tha different achoola 
remained uniform In tha following waya: 

1. Control groupa. in each achool control group* racalved that achool'a 
atandard method of lnatructlon ln gynecologic oncology. In the different 
achoola, the atandard method of lnatructlon varied from a aerlea of 
formal lecture* ln ona achool to dlatrlbutlon of a Hat of racowendad 
reading with a total abaenee of cleaaroom instruction ln gynecologic oncol- 
ogy ln another achool. Thla la ahown ln Tabla I. 

2 * flg | , * rl — nt«l _g roup« , i n ,ii achoola, experimental group* wara given tha 
Unaarly programmed "content" text at tha beginning of tha period of 
lnatructlon. Tha taxta ware collected at tha and of tha period of 
lnatructlon and ahlppad to the Medical Collage of Georgia, in thoaa 
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MCTMODS or INSTRUCTION for five schools in study. 



SCHOOL 


CLASS OF 
szuDons 


INSTRUCTION METHOD 

INSTRUCTIONAL FIRIOD CONTROL CROUPS 




INSTRUCTION METHOD 
EXPERIMENTAL GROUPS 


California College 
of Hadlclna 


Sophoaoraa 


Spring Quartar 


8 lectures 


Gynecologic oncol- 
ogy "content 11 text. 
No lectures. Ques- 
tion end answer 
sessions offered 
once a week. Student 
attendance « none. 


University of 
Nebraska Collaga of 
Hadlclna 


Juniors 


5 waaka 


1-2 houra seminar Gynecologic oncol- 
tuaor taachlng an ogy "content" text, 
clerkahlp 01 emphasis 








(Tha entire claaa attended 6 houra 
of lecturea given once a year.) 


Unlvaralty of North 
Carolina School of 
Hadlclna 


Seniors 


9 weeks 


OB-GYN seminars once 
a week. 

Reading list on 
Oncology 


Gynecologic oncol- 
ogy "content" text. 
Seminars once a 
week - optional. 
Student attendance ■ 
none. 


Stata Unlvaralty of 
Iowa School of 
Hadlclna 


Junlora 


Wi week. 


Joint conference Jr a 
& Sr. students. OB- 
GYN 1 hr. every 2 
weeks. 


Hynoeologle oncol- 
ogy "content" text. 

No conferences or j 

lectures. j 

i 








j 

Teaching clinic 1 hr. 
each day attended by 
Jr. students and GYN 
residents conducted by 
Sr. staff Heather 


Unlvaralty of Verannt 
Collaga of Hadlclna 


Juniors 


12 waaka 


5 -12 hours of 
lecluras In Jr. yr. 
(Clerkship In Sr. 
yr.) 


Gynecologic oncol- 
ogy "content" text. 
No lectures. 


i; 
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schools where control groups received forml clessroon instruction 
in gynecologic oncology, ths dsssss wn onittsd for ths sxpsrinsntsl 
groups. 

3 * .CrltTlon imurti, both arouse. All students participating in ths 
study wars asked to keep a record of their tine spent studying gyneco- 
logic oncology. All students in ell groups were given as a pre-test 
one forn of the Special National Board lamination in Og-OIW Neoplasm 
and a second form of this examination as a post-test at the end of 
the period of instruction. 
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111, Collection of Data , 

A. Su— rv of Evaluation Froiram , 

1, Medical Collate of Geortle . Table 11 summarises the evaluation program 
at the Medical College of Georgia for the two year*. The number of 
■tudenta In each group and the order In which they took the apeclal 
National Board "content" Examinations in OB-GYN Neoplasms are given. 

Study tlae data and oral exaailnatlone ware required of all atudente. 
Attitude surveys, however, were required only of thoee ualng the 
prograaewd text a. The table shove the code numbers of the National 
Board of Medical Examiners, Fart 11, Comprehensive Examination In 
Obstetrics and Gynecology. 

2. Other schools . Table III Indicates the else of the groups at the other 
schools and the order of pro- and post-tests given. Requirements for 
time sheats and attitude surveys are also Indicated. 

B. Administration of Criterion Teeta . 

1. "Content" pr—taete . At the Medical College of Georgia In 1963-64 and 
1964-65 and at the other five schools In 1964-65 a 90-mlnute, 108-questlon 
Special National Board Examination In OB-GYN Neoplasms prepared for the 
project by the National Board of Hidlcal Examiners was administered during 
the first week of the Instruction period to every control and experimental 
group. Of the pair of matched examinations prepared by the National 
Board of Medical Examiners, the test used as a pre-test for one group 
would ba used as a post-test for the following group and vice versa. In 
the second year of the project, the original pair of examinations (A and B) 
ware disguised with scrambled and renumbered questions and Issued as 
examinations G and D. 

2, Administration of "content" pnat-tneta. The 90-minute, 108-quastlon 
post-teats were administered to each experimental and control group 
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TABLE III 

SUMMARY OP EVALUATION PROGRAM 
Pive Schools In .Study 



SCHOOL 



# 

STUDENTS 



California College o£ 
Medicine (divided class 
simultaneous treatment) 



"CONTEND 1 * STUDY 

EXAMS TIME 

Pre-Teat Foat-T eat 



ATTITUDE 

SURVEYS 



It Control 
lb Experimental 



45 

47 



C 

C 



D 

D 



X 

X 



X 



University of Iowa School 
of Medicine 



I 


Control 


10 


XI 


Experimental 


9 


III 


Experimental 


13 


IV 


Control 


10 


V 


Control 


11 


VI 


Experimental 


13 


VII 


Control 


11 


VIII 


Experimental 


11 



A 

B 

B 

A 

A 

D 

C 

D 



B 

A 

A 

B 

B 

C 

D 

C 



X 

X 

X 

X 

X 

X 

X 

X 



X 

X 



X 

X 



Univeraity of Nebraska 
College of Medicine 



I 


Control 


10 


II 


Experimental 


11 


III 


Control 


9 


IV 


Experimental 


10 


V 


Control 


9 


VI 


Experimental 


11 


VII 


Control 


10 


VIII 


Experimental 


10 



A 

D 

A 

B 

A 

D 

G 

D 



* 

A 

B 

A 

B 

C 

D 

G 



X 

X 

X 

X 

X 

X 

X 

X 



X 

X 

X 

X 



Univeraity of North 
Carolina School of 
Medicine 



X Experimental 
II Control 
IXI Experimental 



14 

14 

16 



A 

B 



B 

B 



Did not take 
tests 



Books loot 
X 



No forms 



Univeraity of Vermont 
College of Medicine 



I Control 
IX Experimental 
IXIa Control 
Illb Experimental 



* 



split 



14 

16 

8 

7 



A 

B 

G 

G 



B 

G 

D 

D 



X 

X 

X 

X 



X 

X 



^National Beard Special Examination in OB-GVN Neoplasms, 
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as close to the end of the clinical clerkship or period of instruction 
as other commitments would permit. At the Medical College of Georgia, 
the examination was administered in the eighth week of the clerkship 
in 1963-64 and in the sixth week of each clerkship in 1964-65. In other 
medical schools in 1964-65, the examination was usually administered in 
the last week of the period of instruction. 

3* Administration of oral examinations . 

a. I nstructions , in 1963-64 and 1964-65, at the Medical College of 
Georgia only, structured interviews were conducted by a panel of 
two visiting judges at the end of the second quarter and at the end of 
the fourth quarter of each academic year. During each week-long 
examining session, the Judges conducted interviews with all students 
from both experimental and control groups of the current and the 
preceding academic quarter. Prior to the start of each examining 
session, the judges were oriented verbally and by written instruc- 
tions as to the purpose of the examinations and the procedure to be 
followed. (See Appendix D) 

b* Deviations from instructions . The procedure the Judges actually 
followed during the four examining sessions in the two years of the 
project differed from the instructions'* in the following ways: 
i. The interval between students was set at 30 minutes and 
duration of the interview was standardized at 25 minutes, 
ii. The judges in presenting cases to the students made frequent 
use of colored slides depicting patients and/or pathological 
specimens* 

iii. Tape recordings of interviews wore made and discussions 
were held with the Judges to encourage them to adhere to 
the specified purpose and format of this type of interview. 



fl 
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iv. The judges found that in reaching their final poolsd grade 
"by consensus," it was impossible for them to distinguish 
between "application" and "content." Therefore, the final 
grade was recognized as representing both aspects of the 
student's performance rather than "application" alone. 
c * limited to local study . For obvious reasons, the program of oral 
examinations took place only at the Medical College of Georgia and 
was not replicated in the second year of the project in the other 
medical schools participating in the study. 

4. Administration of "retention" post- tests . 

a. Administration , The instrument for this evaluation was the compre- 
hensive Examination in Obstetrics and Gynecology, Fart II of the 
national Board of Medical Examiners. Hie test, in various forms, 
was administered to an entire class of the Medical College of Georgia 
at the end of May, after the conclusion of the last quarter of the 
academic year. The following classes took the test: 

i. May 1963, the Junior Class. 

ii* Hay 1964, the Junior Class, and the Senior Class, 
ill. May 1965, the Junior Class and the Senior Class. 

Each test consisted of about 150 multiple-choice questions on the 

"content" of obstetrics and gynecology in various categories: 

1. Embryology, Anatomy and Physiology of tho Female Organs of 
Reproduction. 

2. Physiology and Ecology of Woman 

3. Normal Pregnancy: Physiology, Biochemistry, and Psychology, 
Diagnosis, Management. 

4. Physiology and Conduct of Normal Labor and Parturition; the 
Newborn. 

5. The Fuerpcrlum; Normal and Abnormal 

6. Complications of Pregnancy 

7. Complications of Labor And Delivery 

8. Disturbances of Function 

9. Anatomic Pelvic Disorders 

10. Infections 

11. Neoplasms 



b. Special d*ta„provldcd # Hie eleventh category, Neoplssms, consisted 
of about 35 multiple-choice questions. For each class and for each 
control and experimental group participating in the project, the 
National Board of Medical Examiners provided the project with an 
item analysis of the performance of each group or class in each 
category# 

c. Measurement of retention # Retention of 7 earnin'* of content was 
measured by repeating the administration of the test (in a different 
form) to control and experimental groupa a year after their completion 
of the teaching program in obstetrics and gynecology, just before 
their graduation fresa medical school at the end of the senior year# The 
change in score in Category 11, Gynecologic Neoplasms, during the 
one-year interval waa used to measure retention# 

d. Deviation from norms # Hie fact that the mean acorea of afcudenta at the 
Medical College of Georgia were below the national mean scores in 
nearly every category ahould not be interpreted aa a reflection of 
the achievement of learning obatetrics and gynecology at the Medical 
College of Georgia in comparison with the national average# First, 
the students in the project were juniors and took the test as psrt 

of a course requirement# Hie national averages were compiled from 
the scores of senior students who took the test as candidates for 
medical licensure# Hie 'senior students in the project who took the 
test did so neither as candidates for licensure ncr to satisfy course 
requirements. Hie test was administered to senior* is part of the 
research project# Hence, the students had little incentive either to 
study for the test or to make an extra effort during the test to 
make a high score# 0 



Measurement of "application" by written testa (Medical College of Georgia , 

1964-65) . 

a. Pre-teats . Each control and experimental group received, in addition 
to the "content'* pre-test, an "application" pre-test which consisted 
of three "clinical problem-solving (or tab-item) tests, " each of which 

r 

required about a half-hour for the student to complete. The purpose of 
administering these examinations as pre-tests was chf.efly to familiarize 
students with the new and radically different format. There was 
little expectation that the tests would serve as quantitative 
measures of the students’ entering repertory of clinical problem- 
solving skills. 

b. Post-tests . Each control and experimental group was administered 

at about the time of "content" post-test an "applications" post-test 
consisting of five tab-item tests. Students were given three hours 
to complete the five tests. 

c. Test schedule. Clinical problem-solving tests were administered to 
students of the junior class in accordance with Table IV. The 
v ests Identified on the table only by their letter designations 
are described in greater detail in Appendix E, which also includes 
•ample Test "A". 

d. Changes in test s. During the academic year 1964-65, parts of the 
nine tests underwent modification after each use. The correction 

of many defects in the content and format of parts of the tests made 
them function more efficiently, but invalidated comparative scoring 
of parts of the tests which were revised between administrations. At 
the end of the year, only those parts of the tests which remained 
unchanged from one administration to the next were considered suitable 
for evaluation of student performance in specific problem-solving 
skills. 
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XAtJLL XV 



TESTING SCHEDULE FOR CLINICAL PROBLEM-SOLVING (TAB-ITEM) TESTS 
ADMINISTERED TO THE JUNIOR CLASS OF THE MEDICAL COLLEGE OF GEORGIA 

1964-65 



PRE-TESTS 

1st Control Group A (+B) + D' 

1st Experimental Group a* + D 1 + E* 

2nd Experimental Group A + D 1 + E 1 
2nd Control Group A* + d* + E* 



POST-TESTS 
A* +B+C+D+E 
A+B+C+D+E 
A+B+C+D+E 
A+B+C+D+E 
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C. r ipe to Criterion Record! . Prior to the start of the project, the method* 
used to record the student study time in this project were specified. The 
text of the original nemcrandu&i on tine to criterion records and a sample 
time sheet may be found in Appendix F. 

D. Attitude Surveys , Students in all phases of the project who received 
programmed texts for study were required to turn in a complete attitude 
survey at the end of the course. They were permitted and were encouraged to 
fill out the questionnaire anonymously and turn it in unsigned. (See Appendix 
G for a copy of the Survey). 
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CHAPTER POUR 

DEVELOPMENT OP PROGRAMMED TEACHING MATERIALS 



Work Acc omplished Prior to Start of ProlecL . 

A. R e search PUn , In the year preceding the atart of thia project, the reaearch 
plan waa developed in detail. The purpose of the project, to evaluate 
programmed instruction as a means of teaching gynecologic oncology to Junior 
medical students, was decided upon. An acceptable experimental design was 
prepared. A testing program was estaollshed using wherever possible existing 
materials and standard methods. The crucial items in the reaearch plan 
which were not available to the project were the programmed texts. 

B * M * terl,1 » Needed , it waa obvious *hat if the research part of this 

project was to be of any value, the programmed teaching materials to be 
evaluated would have to be of the highest quality possible. No useful 
purpose would be served by setting up an expensive, controlled, balanced 
study only to demonstrate that the best possible clinical teaching by 
conventional methods is superior to hastily contrived, improperly revised, 
and inadequately validated programmed materials. Furthermore, an essential 
part of the research project was to develop methods to permit subject-matter 
experts In medical schools to prepare, revise, and validate programmed 
materials in their specialties with assurance that their completed program 
in its final form would be of satisfactory quality# 

C * ^& E rt . t . y . rc Search . Prior to the start of the project, the literature was 
reviewed on the following areas; 



Is 

2 . 

3 . 

4s 

5 . 

6 . 



Response mode (multiple-choice versus constructed response) 

Step size (small steps versus large steps), * 

Density (ratio of new responses to total responses), 
Presentation mode (teaching machine versus programmed text). 
Page format (horizontal versus vertical), and 
Programming strategy (linear versus branching versus composite). 
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Thie review led to the conclusion that each of these was a complex 
problem for which there was at the time no categorical answer which could 
be relied upon for clinical teaching in medical schools. The original 
grant proposal presented the problem this way; 

M Thc most difficult variable to control and evaluate in interpreting 
results of programmed instruction research is the quality of the 
program. This seems to be an inherent problem. The effectiveness 
of an experimental programming technique will vary with the quality 
of the programming. By the time one has established objective 
criteria for quality control of the program, one has usually also 
answered some of the problems of the effectiveness of the programing 
techniques." 

Oy.i&inai Working Principles. The working principles for the development of 
programmed texts in this project were based largely on the experience of 
others (sea Bibliography). The working rules were originally only 6. 

1. Use programmed text rather than teaching machine presentation. 

2. Use dense linear programs to present both new and remedial material. 

3. Use branching format primarily to diagnose the student's 
deficiencies. 

4. Use clinical applications to help motivate the student. 

5. Use constructed response whenever a complex verbal or diagrammatic 
response is required. 

6. Restrict multiple-choice answers to situations where the student 
has indeed a choice to make. 

Two additional principles were added after tryouts of the first drafts of 
portions of the linear text: 

7. Use thematic rafchor than formal prompting. 

8. A\oid typographical cueing. 
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II. Control* of Courts Content . 

Before any of the teaching aateriala wera developed for the project, two Method* 
were developed to limit or define the courae content to be covered in the pro- 
gramed text and in the special lectures prepared for control groups. 

A. Agreement on Courae Requirement *. 

1 * "Raqulrcmants of coursa." A« * guide for both the lecturer end the 
prcgreA writer*, a 2500 word outline of the "Requiraasnts of Courac" 
wee prepared (See Appendix 11). This outline attempted to define the 
verbal knowledge and skills in gynecologic oncology which a {radical 
studsnt waa expected to be able to demonstrate at the end of the 
"course of study* 11 Hie M courae of study" was assumed to include, in 
addition to didactic instruction and outside reading, appropriate 
clinical experience in the care of patients* Hie "Requirements" rep- 
resented the skill and knowledge in gynecologic oncology which the 
student was expected to acquire aa a result of his total experience 
in the clinical clerkship, rather than as a result of his learning 
from a specified instructional method* 

2# Selection of topics. Preparation of tho outline took place over a six 
months’ period prior to the start of the project* Many drafts of the 
outline were reviewed and revised by different faculty members of 
the Department of Obstetrics and Gynecology of the Medical College of 

a 

Georgia. Controversial items which were rejected by eome faculty 
members and recommended by othere were retained in the outline. 

3. Effort at behavioral term*. Although much effort w*s expended in trying 
to define the requirement* of the couree in behavioral term*, the final 
product seemed to represent e consensus of the teaching mime of depart- 
mental faculty members, without regard to the restrictions Imposed by 
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time, by the limited availability of facilities and paticnto, and by 
the personal limitations of students and faculty members. Thus, the 
outline was not so much a set of behavioral objectives for the progranwed 
text as it was a guide to students of the knowledge and skills which 
faculty members in the Department of Obstetrics and Gynecology expected 
them to have achieved at the conclusion of the course of study. Although 
the language of the "Requirements of Course" expressed the course 
objectives in terms which were often non-bchavioral , the outline never- 
theless served as a useful and meaningful guide to both the lecturer 
and the program writer in preparing their teaching materials. 

®* Agreement on Degree of Detail. The writers of the programmed text and the 
lecturer agreed tc use the "Requirements of Course" aa the specific outline 
of diseases and disorders which would he presented in the two teaching 
methods. They also agreed on the degree of detail they would present in 
accordance with the principles set forth in the experimental design of the 
project (Sec Appendix F). 

HI.. Development of the "Content" Text . 

A * PMCE.lptlon of Text. The "content" text in its final form is a non-Skinnorian 
linear text of 830 frames with 45 illustrations or diagrams. Nearly all 
frames require several constructed responses. The text is presented in a 
horizontal format with two frames to the page. Unconventional features of 
the text result from the assumptions which were made. 

B* Assumptions of the Program Writer. 

1. Uaa, The program would be adjunctive rathar than inclusive in its 

coveraga of contant. It would be written with the admittedly erroneous 
assumption that the student entering his junior year of medical school 
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had available*for inatant recall, without special prompting, 
all his knowledge of gross and microscopic anatomy, general and systemic 
pathology, physical diagnosis, pertinent aspects of biochemistry, 
endocrinology, radloblology and related basic medical sciences as they 
apply to the study of gynecologic tumors. It was recognized that gyne- 
cologic oncology Is largely a series of spsclal applications of knowledge 
to which the medical student has had some exposure In his freshen and 
sophomore courses. Therefore , most of the programmed text would be 
made up of frames requiring the student to make new applications of old 
(and largely forgotten) knowledge, in the programmed text, the basic 
sclonco Information which the atudent was assumed to have learned 
previously would be presented to him to be re-learned only after he had 
demonstrated to himself that on his own, he was unable to supply this 
Information In Its specific application to the problem under consideration. 

2. Density, Repetition would be scanty and widely spaced and would be 
Introduced into the text only in response to repeated demands by students. 

3. Prompts. The single-concept frame would be used sparingly. Many frames 
would present multiple blanks and multiple Interrelated ideas simul- 
taneously. Ihis would permit greater use of thematic rather than formal 
prompting, and would further reduce the nead for repetition. 

4. Difficulty , in preaenting new and unfamiliar material to the atudenta, 
the program would demand thft the atudent use intuition us well as 
Ingenuity in guessing what the next step might be. Whenever possible, 
the student would be asked to write out a new word and use it correctly 
or express a now concept before, rather than aftei> it was presented to 
him for the first time In print. 

£SS8£ll« Hie programmed text would be shorter and more concise than 
conventional prose treatments of the same subject matter in ordinary 
textbooks. 
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C# Studont Acceptability . It was essential to the success of this project that 

the programmed text in tlio final form be favorably received by tho students. 

1. Challenging, it would have to incur the student's respect. It was 
assumed that this could be accomplished by maintaining an element of 
intellectual challenge. 

2. Efficient . It would have to appear to be a short-cut to efficient learning. 
It was assumed that this could bo accomplished by keeping tho text as 
short as possible. Brevity could be achieved by strict adherence to the 
"Requirements of Courso" and by eliminating from tho text the re-teaching 
of basic science in 'creation which tho student might have already mastered. 

D. First Draft of "Content" Text . 

1. Rescript: ion , Hie programmed text as it was originally drafted resembled 
an examination of nearly a thousand questions calling for nearly all tho 
information specified in the "Requirements of the Course." The questions 
required written answers, 'ill ere was no repetition except for 
occasional summary or terminal frames. There were no illustrations and 
very few prompts or cues. It was obvious that tho text was less than 
perfect as an effeetivo and acceptable teaching instrument for medical 
students. 

2. Tho defects, had a virtue. Hie programmed text in its first draft was 
too short, too demanding^ and too complex for a student who was not 
already a master of the subject matter. Hie errors and excesses of tho 
program in its first draft, however, were all in tho same direction. They 
could bo detected and overcome in the process of rovision. it was expected 
that trial students working through the program would protest 

against* the excessive and unreasonable demands made of them and would 
be able to suggest specific retaedieo on a frame-by-frame basis. Hie 
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writer recognized that excesses and errors in the other direction, 
making the program too long, too simple, too boring and repetitious, 
would be far more difficult to correct during the process of revision. 
Trial students faced with such a program might express generally negative 
reactions to the text but would bo unlikely to make specific, frame- 
by-frame suggestions to eorreet those errors and excesses. The program 
writer, by ignoring the needs of students for prompts, cues, and 
repetition, was able to put the entire text on paper in a short 
tioo and limited his responsibility to specifying what the 
text was supposed to teach. Ms predictions and assumptions of what 
a Junior medical student knew or did not know, or of what ho found 
difficult and what he found easy wore too unreliable to bo of value 
in making the first draft of the program, 'flic result was a very 
difficult text which, in ways the writer could not predict, required 
extensive revision by "export consultants. " 

IV. Revision of "Content" Text . 

A * gonsMlfrintn Kr^loyed. in the revision of this program, tho "export learning 
consultants" were a carefully selected group of senior and sophomore 
nodical studonts. Also, certain faculty members volunteered to work their 
way through parts of tho text in a specified order. The student consultants 
received a small hourly remuneration for their work. 

B. Procedure for Making Revisions'. When a consultant completed a section of 
the text, all frames in this section requiring revision were rewritten and 
retyped before the text was presented to the next consultant, who thus 
saw only fresh unmarked copy. Some frames wore replaced as many as five 
times. When tho final program was duplicated for operational uso in tho 
project, seareoly a frame was left from die original draft of the program. 
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Description of Consultants , The following consultant* were u*ed: 



1# Senior student* An exceptionally able and Mature senior medical 

student who was already familiar with the subject matter was first. Tills 
student pointed out the most glaring errors* non sequlturs* and 
unreasonable obstacles in the program. His corrections made the 
text smooth enough for review by a subject-matter expert. 

2« Faculty member * Fellow faculty members who were experts in the 

subject matter followed* Different faculty members reviewed different 
sections* reviewing subjects of their special interest or competence. The 
purpose of this revision was to detect errors and deficiencies in the 
content. The suggestions of these subject-matter experts with regard to 
format or programming technique were courteously received but were 
seldom incorporated into the revised text. 

3, Sophomore student . An exceptionally able student who was unfamiliar 
with the subject natter but had an excellent academic record and good 
background preparation came next. Tills student by suggesting revisions, 
repetitions* additional cue*, illustrations, sumary frames, and changes 
in the foL'Gat provided information which permitted the program to 
function rather well as a self-instructional tcx fJ for other well- 
prepared and well-motivated students. 

4. Sophomore student . After this came a student of less than average ability 
who was unfamiliar with the subject matter and whoso preparation for it 
was at beat uneven, but who was conscientious and well-motivated. Hio 
revisions suggested by this student included additional repetition, 
additional cues, and more illustrations * and other specifics changes. 

The effect of his suggested revisions was to render the text usable 
by the least prepared students in the class. 
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5. .Sgnior qtudant. Tfca final consultant ve* a asnior student whose 
familiarity with tba subject had long since evaporated and whose 
interests lay In other araaa* This disinterested^ poorly Motivated 
student tended to skip Material which bored him end to dodge the challenge 
of Materiel which he considered unnecessarily difficult. This student 
was contcnptuous of the subject Matter and of the method of presenting 
^ made an effort to find something to criticise on every pege. 

The end result was a far more polished program than it would have been 
otherwise. 

D* Effect of Conmlrente on Text . 

l * £**£ e ffective . The critic lams of the first three 

consultants permitted revisions of the text which changed is from an 
ineffective teaching instrument into sn effective, if somewhat rough, 
programmed text. It is doubtful whether the fur flier revisions 
improved the effecti vynpBs of the text as a teaching instrument. 

2 * @«ZJMde_tlie. text acceptable. Hie criticisms of the last two con- 
sultants were of value chiefly in improving the acceptability of the 
text to medical atudents. In particular, the text was made more 
acceptable for ill-prepared and poorly Motivated students in the claas. 

It is probable that some of the suggestions of tho last two consultants 
for additional repetition and more explicit prompts tended to blunt 
tho intellectual challenge ,o£ parts of tho text and perhaps made the 
text less acceptable to tho academic leaders of tho class. Gnch an 
offect should ho considered unfortunate; the leaders, of all students 
in the class, ware perhaps least in netd of programmed instruction to 
help then learn; nevertheless, their position as leaders made it impor- 
tant that tho text receive their endorsement rs well. 
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Information frames# If the student selects an option that permits 
him to gather more information about the patient, he may be 
referred to frames which provide him with information about the 
patient's history, general physical examination, special examina- 
tions and various types of laboratory data and diagnostic procedures. 
These pages may consist of a prose paragraph of information, or, if 
more active participation by the student is desired, the page can be 
a coded data-gathering frame consisting of a numbered list of items 
about which the student is expected to want additional information. 
Each of thos*> items is numbered in scrambled order. Adjacent to 
this list, there' is a numbered list of ’hnswers" (including physical 
findings, laboratory reporLs, and other data) presented in numeri- 
cal order. The student must select each item about which he wishes 
information and find the answer with the corresponding code number 
on the adjacent list, lie thus must identify specifically each 
item about which he wants further information before he is given 
tlie answer. 

L '’ Purpose of coded information frames . The case presentation using 
coded data-gathering frames volunteers no summaries of information 
and in this way resembles the patient, the physical examination, 
and the laboratory. The student geLs only the information he 
specifically seeks. In such case presentations, as in evaluations 
of real patients, there may be more than 100 items about which the 
student has the option of seeking further information. Most of 
these contribute almost nothing to the student's understanding 
of the patient's problem. Only by seeking the information, however, 
can the student learn which information is relevant. Afte*- he 
makes his choices, he can be informed by a simple code whether or 
not his choices were considered appropriate. 



0 

Q 

Q 



0 

a 



o 



o 

a 

D 



tv v 



POOR ORIGINAL QUALITY -BEST COPY AVAILABLE 



o 

ERiC 




-53- 



2. Model o f management program. 

a * £ ° n8trUCted res P° nse £rame8 - Figure II shows the general scheme 
for presentation of management problems. In Frame 1, the student 
is asked a question which requires a written response. Hie question 
is often no more than a request to "please write down your next step." 
Some students are stumped by a question like this and fail to write 
down anything at all. When this occurs, the student is referred to 

an explanatory frame (Frame 2) which suggests a general approach which 
should be acceptable. 

b. D irectory frames . After the student has written down his response, 
he turns to the next frame (Frame 3), which consists of a list of 

possible answers which he may have written, and a directory of where 
to proceed next. 

C# frames . If the student^ answer is so unusual as not to 

be listed, he is referred to Frame 4, which explains that he 
overlooked the proper response and refers him to Frame 11 for 
remedial advice. This remedial advice may consist of a list of 
outside reading, or a referral to specific parts of other programed 
materials such as the "content" text or a referral to the data- 

gathering frames of this case presentation, depending on the nature 
of the error. 

d. Explanation frames. Often the student will have written a wrong 
answer which has already been anticipated in the program and is 
listed among the multiple choices. Each of these listed wrong 
answers has its own explanatory frame which defines the student's 
error specifically and then refers him elsewhere for remedial 
advice or instruction. 
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El 3&fi J te £SHiai4gaL « In the revision of the programmed text, great care 
was taken to insure that the consultants did not express their criticisms 
directly to the program writer. Instead, the criticisms were either 
expressed In writing or were expressed verbally to a third person who 
wrote them down. It was essential that the third person be one to ’whom 
the students could talk freely, revealing both their own Ignorance and 
the Inadequacies of the programmed text as a teaching device. The Inter- 
position of an Intermediary between the student consultants and the 
program writer facilitated the process of revision In two ways. 

1. Student freedom. The student consultants were protected from being 
judged by the program writer (one of their teachers). They were 
encouraged, by a person who knew less of the subject matter than they, 
to express their criticisms with as much hostility and frankness as 
possible. Without this protected, permissive situation many 
students would not express themselves freely. The result was a 
wealth of critical comments which might not have been otherwise expressed. 

2 * — r - ter restrictions. The program writer was permitted to teach only 

by means of his paper program. Whatever the reaction of the students, 
he had no opportunity in person to clarify misunderstandings or to 
justify his work. Only by changing the program until the students' 
responses to It were satisfactory could he function effectively u b 
a teachers 

Pl lL«g-0£ Student and Faculty Time. The preparation of the "content" text In 
gynecologic oncology for use at the Medical College of Georgia in the 
academic year 1963-64 was accomplished In three months (June, August, and 
September 1963). No exact record of time expenditures was kept for any 
personnel participating In the production except the students (who were 
paid on an hourly basis). Nevertheless, the following estimates may be of 
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a three -month period). 












2. Other subject-matter experts (4) - two hours each. 

3. Student consultants (equivalent to 4) - 25 hours each. 

4. Intermediary - 50 hours. 



5. Medical Illustrator - 25 hours. 

The above figures do not include time spent preparing the "Requirements of 
the Course" or supervising the process of duplicating the finished text. 

G ’ gg v lslpn of "Content" Text after Field Testing i n the iprlng of 1S64 , 
tally was made of al x responses In 18 randomly selected programmed texts 
which had been completed by students In the experimental group, of the project 
during the year. It was hoped that such a tally would bo helpful In provid- 
ing guidelines to permit revision to be made on an objective basis. These 
expectations were not fulfilled. 

1* Determination of error rate . 

a. Method. From a randomly selected .ample of 25 consecutive frames 
(numbers 76 to 100) an error rate was determined by dividing the 
number of possible errors that could have bean made In the 25 
frames by the number that actually were made. The figure for 

the 25 frames was 77.. Th 2 frame-by-frame error rate, however, ranged 
from 07. to 28£. (See Table V). 

b. Interpretation. A careful study of the response In this sample 
sequence o£ 25 frames was compared with responses to suossary or 
terminal frames further on In the text. It became apparent that 
the effectiveness of a frame as a teaching device was not 
correlated with a low error rate. In fact, it could be 
demonstrated by the high incidence of correct responses later In 
the text thet some of the most effective teaching frames In the 
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76 

77 

78 

79 

80 
81 
82 

83 

84 

85 

86 

87 

88 

89 

90 

91 

92 

93 

94 

95 

96 

97 

98 

99 
100 

TOTALS 



BLANKS 

2 

2 

5 

1 

2 

2 

1 

3 

4 
2 
1 
X 
1 + 

2 

3 

3 

1 + 

1 

1 

2 

2 

1 

2 

2 

2 

49 



ERRORS 

0 

10 

5 

5 

5 

10 

0 

11 

2 

2 

0 

X 

0 

2 

0 

0 

0 

1 

0 

0 

7 

1 

0 

0 

2 

63 



POSSIBLE ERRORS 

36 

36 

90 

18 

54 

36 

18 

54 

72 

36 

18 

X 

18+ 

36 

54 

54 

18+ 

18 
18 
36 
36 
18 
36 
36 
_ 36 

882 



ERROR RATE 
0 

• 28 
.06 
.28 
.09 
.28 
0 

.20 

.03 

.06 

0 

X 

0 

.06 

0 

0 

0 

.06 

0 

0 

.19 

.06 

0 

0 

.06 



.07 
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ZABLE V 

SUMMARY OF ERROR RAZES 

Z.llle. of answer, from 18 junior 
medical students on Frames 76-100 
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NOTE: 1+ means "answer in your own words." 
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sequence were those with very high error rates. On the other 
hand, it was also apparent that some of the frames which elicited 
error-free responses from all students were also highly effective 
as teaching frames and could not be eliminated on the grounds 
of rtdundancy. 

c. v alidity . The problem of calculating the error rate in a progressed 
text which does not insist upon a precise vocabulary but requires 
the student to find his own words ij in itself a problem which 
renders the calculation of error rates almost as meaninglese 
as their interpretation. The responses to Frame 293 illustrate 
the problem and are shown in Table VI. 

Revised . Although the students' error rates proved of little 
value in detecting the frames requiring revision, their responses and 
sometimes their marginal remarks were most helpful. Most of the frames 
requiring revision were in one or more of the following categories. 

a. Frames using typographical cueing. 

b. Frames with ambiguous instructions which left the student 
uncertain of the type of response he was expected to make. 

c. Frames containing factual or "content" errors. 

d. Frames containing overly dogmatic statements requiring qualifying 
words and phrases. 

The first year's revisions -were made with the intent of making the text 
mort acceptable to casual inspection by subject-matter experts (faculty 
members) than of making it more effective as a teaching instrument 
for students. In all, about 257. of the frames received soma modification, 
ranging from complete rewriting to minor typographical corrections. 
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TABLE VI 



LISTING OF RESPONSES IP OPEN-END FRAME 



Examples of answers given by 18 students 
choeen at random to Frame 293 



Frame 293 - 



Question: In Stage 
cervix." Why, then, 
cures? (Your Words) 



I cancer, the "cancer is strictly confined to the 
doesn't a simple total hysterectony result in 1007, 



Recommended Response (concealed): 



(in effect) 

THE STAGING IS BASED ON CLINICAL EXAMINATION ALONE. 
The clinical examination is frequently erroneous as 
an estimate of the true extent of the disease. 
Actually, about 20% of Stage I cases have spread 
beyond the cervix, but this is not detectable by 
pelvic examination. 



Student Responses; 



2 1 
*3 1 

4s 

*5. 

6. 

7. 

8. 

9. 

10 . 

11 . 



12 . 

13. 

14. 

15. 

16. 

17. 

18. 



Bmm dlil»« wtattMM vi. lymphatic. 1. possible; the staging is by 
clinical evaluation alone and docs not rule metastasis out. 

"“V 0 ? 7, co f r « ct «nd many Stage II ca's are dx. as Stage I. 
C f* J“ ltlcentrlc ir > origin and other sites are present though not 
clinically detectable. 1 

Due to error of dx. of the stage. 

Every woman does not gst yearly Pap smears. 

Because of contamination during removal. 

Not really Stage I - some extension or metastasis has occurred. 

Inadequate excision. 

Staging is based on clinical evidence, actual extent of disease may be 
|rca ter. 

Invasion of lymphatics can occur with distant metastases early in the 
course of stage I. 7 1,1,13 

2l C W4B wron * " l<lc c ** invaded parametria, etc. 

(bj. The highly malignant epithelium developed another center of Ca. 

Clinical diagnosis, not pathologic. 

potentiaI* 1U * tl0n °* Bt *** " aultlple o£ pw-Mlignancy - malignant 

Inadequate Dx. 

^ »» «* te " ilv « than that se.n only by early clinical exam. 

Can t b« that definite in statement like included in quotation. 

involvement 11 " 1 b * Md ° n cllnlcal iB PM..ion and this may differ from actual 
* clinical impression and may not be pathologically true. 



*considsr«d inadequate answer. 
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H. R evision of "gonigi . t'' . AL end of second Tear of M,ld Teat In. - th , 
■umoer of 1965, after the text had been used by several hundred Mdlcal 
student. In six different medical schools, tallied were «d« of all response, 
from 25 texts including samplea from all schools participating in the 
study. Forty frames and three illustrations were identified as containing 

error, which required revision. The error, in th. 40 frames were classified 
as follows: 

1. Typographical errors, 8 

2. Errors of fact or substance, 30 

3. Errors in programming technique, 2. 

In this final revision, changes were made in less than 5 X of the frames. In 
this stage of the program development, evaluation of student responses was of 
negligible value in identifying frames needing edifications. 

V. Development of "Applications 11 Text . 

In January 1964, work was started on the second phase of the project, the 
development of a patient-oriented programmed text designed specifically to help 
the student develop skills of clinical problem aolving for patient care. This 
type of programming presented considerable theoretical and technical difficulties, 
A * fi* ner>1 Al P- fCtl of Clinical Pr o blem Solving . Clinical problem solving is 
an extremely complex process. It involves not only numrous interactions 
between the physician and his patient, but also interactions between the 
physician and meny different laboratory services and their personnel. Often 
the process fer Just one patient extends over meny days and involves multiple 
exchanges of complicated information before a resolution of the problem eaa 
be achieved. To try to reduce this complex and variable process to a 
schematic form which can be presented in its entirety on a few sheets of 
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par- , • * certain to result in over-eimpllficatior.. Nevertheless, it *« a »ed 
pro-able that the diatortiona and omlaalona of presenting clinical problem 
aolving through the medium of programmed instruction would at leaat be 
different from and perhaps of leaaer magnitude than the distortion and 
oraiBiloni of preaenting the process through more conventional educational 
media. It was recognised that programmed instruction could provide a 
close parallel to the actual process of clinical problem solving in some 
of the following ways: 

1. The student could be required to begin by gathering information about the 
patient. The more information he gather. d, the more selective he would 
have to oecome in the acquisition of further information. Also, ha 
would have to call upon his fund of specialized medical knowledge and 
apply It to the patient 1 * problems 

2. The student could be required to decide when ha hod enough infox *_tion 

to proceed with the management of the patient, lha answer to the question 
' How much information is enough?", obviously would vary with the 
circumstances. At one extreme, there are clinical problem* where the 
entire process is one of gathering information and correlating it. 

Once the diagnosis i. establi.had and tha patient's problem or problems 
ara defined, further treatment may t* uaneceesary. A t tha othar 
extreme, there arc emergency situations, cardiac arraat, for example, in 
which only the briefest information gathering is possible before 
active therapy must be started. Thereafter, obtaining furthar infor- 
mation, evaluating it, and modifying the plan of traatmant must ba 
carried iorward almuitanaoualy. 

3. The student could ba required to formulate a sequenced plan of therapy, 

and to modify this plan in accordance with changes in tha patient's 
statue. 
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*' gS aw “ r Ba,ed SSSSSSSS^ j- At the outset, it was recognised that if 
programmed material! were to repraeent problems of patient management 
realistically, the medium of presentatlcn must not only permit the 
accumulation of detailed information in an ord.rly fashion, but must 
ulao permit bypassing of information-gathering whenever necessary, it 
was apparent that a computer would be moec ueaful in providing storage 
capacity and random and controlled access to clinical information. 

It was recognized, however that even a computer must be programmed, and to 
program . computer with a clinical problem, a clinician mut jo tha 
programming, it wae further recognized that a computar-based program would 
firet have to be prepared on paper. It wae decided that the project would 
restrict itself to this first stage, that tha case presentations dav.loped 
for the prolect would not reach a degree o£ complexity and sophistication 
requiring the flexibility and storage capacity of tha computer. Tha princi- 
Pla. underlying tha presentation of clinical cases in a paper program, however, 

can aleo be ueed ae a guide for the fu' ire development of coi*mter-baead 
program*. 

C. d eve lo p ment of A Wod a l^ Caee. In the initial stage, of developing the 
case presentation taxt, it became nece.i.ary tu d.velop . programing 
format which would be sufficiently flexible to meet the ineiitutionel 
requirements of the course end yet would provide a defined etructure to 
permit the programmer to orient end Justify the component parts of the 
cesi presentations to bt developed. Tht following model wee ueed me a 
guide in tha davelopment of the case plantation* in tha text. 
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Mp_dcl_of a cage presentation , 

*• fionstructcd-rerpops e frames . Figure I illustrates schematically the 
initial phases of a case presentation. The case begins with a 
summary of the Chief Complaint, and of the Presenting lilt. ess. The 
statement is followed by an open-end question requiring a written 
response such as "Hhat will your next step in management be?". The 
student Is asked to write his response and then turn to the next page 
which gives him a number of options. 

b * Directory .frames^ The student is given a choice to obtain additional 
history, perform a general physical examination, get more laboratory 
data, or, if he thinks the patient's circumstances warrant it, to 
bypass further data-gathering and proceed directly into a plan of 
active treatment. The directory frame may offer him his choice of 
several different treatment plans. 

c * — rjose of constructed-r esponse frames . The purpose of the initial 
question "Hhat will your next step be?" is to encourage the student to 
find this step on his own, without the help of the checklist of 
multiple-choice options given in a directory frame. If the student's 
constructed response differs from ell the options available to him in 
the directory frame, later revisions of the text can be revised so as 
to Incorporate the student's response into the test of options, with 
branches leading to further frames, as appropriate. In a series of 
case presentations, remedial materials are sequenced so that by the 
time the student reaches later, more so*' isticated cases, the dis- 
crepancies between his constructed responses and the options he is 
given tend to disappear. 
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d ‘ Information frames. If the student selects an option that permits 
him to gather more information about the patient, he may be 
referred to frames which provide him with information about the 
patient’s history, general physical examination, special examina- 
tions and various types of laboratory data and diagnostic procedures. 
These pages may consist of a prose paragraph of information, or, if 
more active participation by the student is desired, the page can be 
a coded data-ga the ring frame consisting of a numbered list of items 
about which the student is expected to want additional information. 
Each of thes<» items is numbered in scrambled order. Adjacent to 
this list, there is a numbered list of'hnswers" (including physical 
findings, laboratory reports, and other data) presented in numeri- 
cal order. The student must select each item about which he wishes 
information and find the answer with the corresponding code number 
on the adjacent list. Ik- thus must identify specifically each 
item about which he wants further information before he is given 
the answer. 

'* F M r P oso ° r coded information frames . The case presentation using 
coded data-gathering frames volunteers no summaries of information 
and in this way resembles the patient, the physical examination, 
and the laboratory. The student gets only the information he 
specifically seeks. In such case presentations, as in evaluations 
of real patients, there may be more than 100 items about which the 
student has the option of seeking further information. Most of 
these contribute almost nothing to the student’s understanding 
of the patient's problem. Only by seeking the information, however, 
can the student learn which information is relevant. Afte- he 
makes his choices, he can be informed by a simple code whether or 
not his choices were considered appropriate. 
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2 * Model of management program. 

31 S ^tructed response frame, . Figure II shows the general scheme 
for presentation of management problems. In Frame 1, the student 
is asked a question which requires a written response. The question 
is often no more than a request to "please write down your next step." 
Some students are stumped by a question like this and fail to write 
down anything at all. when this occurs, the student is referred to 

an explanatory frame (Frame 2) which suggests a general approach which 
should be acceptable. 

•». D irectory fra m e s. After the student has written down his response, 
he turns to the next frame (Frame 3), which consists of a list of 

possible answers which he may have written, and a directory of where 
to proceed next. 

C. Re medial frames , if the student's answer is so unusual as not to 
be listed, he is referred to Frame 4, which explains that he 
overlooked the proper response and refers him to Frame 11 for 
remedial advice. This remedial advice may consist of a list of 
outside reading, or a referral to specific parts of other programmed 
materials such as the "content" text or a referral to the dats- 

gathering frames of this case presentation, depending on the nature 
of the error. 

d. Explanation frames, often the student will have written a wrong 
answer which has already been anticipated in the program and is 
listed among the multiple choices. Each of these listed wrong 
answers has its own explanatory frame which defines the student's 
error specifically and then refers him elsewhere for remedial 
advice or instruction. 
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• • Pr ovisional frames . The student «sy write an answer which Is 
partially correct or provisionally acceptable. Under such 
circumstances, he may be referred to a frame such as Frame 7 which 
consists of a clarifying question, usually requiring a written 
answer. If he answers It correctly, he will be referred directly 
to the next decisive question and can proceed In the same way as 
If he had made a fully correct response In the beginning. If 
he falls to answer the clarifying question satisfactorily, he Is 
referred for remedial Instruction (Frame 10). 

3 . Model of r« Bui ti . 

Outcome for patient . Sooner or later the student reaches the end of 
his program of management and Is Informed of the outcome for the patient. 

If he managed the problem appropriately, he Is told eo. If he has 

managed the problem Inappropriately, he Is told the consequences for L 

the patient and the nature of his Inadequacies. This Is Indicated 

schematically irf Figure HI, He 1. also given remedial advice as to 

how best to correct this error. 

4. Model of summary questions . 

Summary frames . The student who has managed the case successfully 
then may proceed to a series of frames which ask him to summsrlse the 
principles upon which he based his correct decision. These decisions 
also call for general "theoretical," "basic science," or other Infor- 
mation of the sort which nay be related only Indirectly to his decisions 
In treating the patient. These summary frames shown dlagraasatlcally l n 
Figure IV can be used to test the student's "book knowledge." j 

D. Selection of Case Material . J 

I 

1 * ^ 1 ~ 1 . t * tlon, _ of The case presentation format selected for the ! 

project seemed sufficiently flexible and comprehensive to permit achl«ve« ! 
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mcnt of all the* teaching aim# specified in •'Requirements of tin- Course" 
if sufficient tiie were available , -me research requirements of the 
project, however, dictated that the time stuilei.ts spent learning 
gynecologic oncology by tien-is of programmed instruction be comparable 
to and preferably leas than the time invested by control groups In 
learning gynecologic oncology from conventional materials. Hence, in 
■electing materials for case presentation programming, the most 
important objectives specified in "Requirements of the Course" wen- 
given preference over some of the more esoteric ones, especially the 
esoteric ones which would have required an excessive expenditure of 
programming time to insure adequate criterion performance by the 
students. 

ifc 1 . t . frl Hir P 0 » c Ic was recognised that the material pr->s.n*td in 

the "content" text could b« represented in case pn-unt it ion format 
by a hundred half-hour case pro vm.tt ions Hi hoot redundancy, i.. 
reduce the number of c isos to .1 ruiog.ail mm or, the teaching objectives 
eif the "Requirements of the Course" lid to Fu- combined |n;.« the 
fewest number of eases p. sslblt ■ il us, , aeb ease present. ,1 or represent- 
ed a numlK r m different diieas.s. . link it pro’ 1 -is, or 1 .. idling 
objectives. Although each vast Ini • fat. oil basis lea r«al life- 
patient, frequently tbs problem, oi sev.ral r.al Us. |-ui. nts would 
he- combined ln;o a single- ease pruo e*ot l.>i provl.’.l tint this con hi 
be done wit lu-u t .iraioi-g the crednlft- ... tin* student, 
j^thod uf selecting eases. Kefore the actual frame writing began, a 
brief a-nopsia was written of each case presentation considered tor 
Inclusion in the program. The avnopg.-s ».vr«* docked, alnglv hi! In 
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combination, with the "Requirements of the Courae," and alao with the correa 
ponding framea of the ''content'* text. Modificatione and transformations 
in the synopses were made to provide th« moat comprehensive coverage 
poaalblc conaiatent with clinical realiaai. 
jfomne* of caae orese nutlons for Initial trveut. . 
a. . Sequence opt tony . Each of the 35 case presentation! prepared for 
the project was designed to represent specific cases of diseases 
and conditions which were covered in corresponding sections of 
the "content" test, flic cases not only differed from each other 
In the "content" they represented, they also differed greatly In 
the complexity and sophistication of the programming techniques they 
crtployed, and In the hinds and degrees of diagnostic and therapeutic 
■kills they demanded of the student. It was possible to arrange 
the case presentations in at least three orderly sequences. 

1. An order corresponding to tho order of diseases and 
conditions presented In the "content" text, 
il. An order representing Increasing coaplexlty and sophlstl> 
cation of programming methodology. 

111. An order representing progressively Increasing demands on 
the student's diagnostic and therapeutic knowledge and 
skill. 

b. Trla|_ M quence. For the initial tryouts, it was elected to present 
the case presentations to the students In an order corresponding to 
that^of the coverage of the same sMterlal In the "content" text. 

Thle was done for the following reasons: 
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i. It permitted the student without prior knowledge of 

the subject nutter to master "content" and the "application" 
of the knowledge In an orderly sequence. If another order 
of case* presentations had besn chosen, It would have been 
necessary for the trial student to master all the material 
of the "content" text so as to prepare himself for the 
challenge of a case which might require hit) to diagnose 

and treat a patient with any disease covered In the "content" 
text. 

11. It rendered the "content" of each cas.- pr«* nt.u ion pre- 
dictable. Tin- student could prepare bin... It for the 
diagnosis anil treatment of the patient simply bv working 

bis way through the next sequence of franc*, in tile "content" 
text. 

ill. With the problems of "content" de-emphasl.tid, it was 

possible for the trial student to concentrate bis attention 
and his criticisms on the problems of format and programming 
in the case presentation itself rather than on the "content" 
dsmands that the case presentation required of him. 

VI. Revision of "Applications" Text. 

A * An ticipated Proble ms of Rev ising Case Presentations . It w , g recognised that 
the revision of the esse presentations, with their branching format, would 
rsqulrc the use of many more student consultants than were nesded in the 
revision of the llnsar "content" text. The process used for the llnsar 
text might serve very well for revision and validation of the "right answer" 
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branches of each ca»e presentation; but only by employing a large 
number of students varying greatly in their preparation for and personal 
approaches to the problem could one identify, revise and validate 
appropriate "wrong answer" branches. In each case presentation it would 
be necessary to identify the following: 

1. Neaded option# (branches) which had bean omitted by the progranmor. 

2. Neaded options which failed to function properly because of 
Inadequate or inappropriate programming teciinique. 

J. Unnecessary options which were never selected by any student and 
could ho delat'd frofi tlu* 

inn s o,j 2 It v,s recognised that the frequency 
with vJucl. an option -a.. wliu.d at. poor index ot its value to the 
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progrin. All opt i»*.i might •«. .i 1 » ted by oily one student out o£ 50 and 
yet ii h.s remedial « null saved bin in.., perpetuating a crucial error, such 
an optioi night be eoitli rctaiuin ; in t; K program. At the other extreme, 
an option frequent l.: selected in students might he no more than an alternate 
branch, without a didoe-tic or remedial purpose, itteluded in the text only 
to Increase tlu .n lout's illusion that i.< tas being given the maximum 
tree turn poesihlt- in r sana -in’, the |Miiott. It was ..Iso recognized that 
a i.roo • ni.ii - r ' '.ranch leading to a iei::-dial -e p:o - u e in tlx "content" 
t--:t night in- pr > ;raiwkd peMectlv .sad v. t t.sf ? to fulfill its remedial 
purpose became ot defied ive pror cars d u ■ in th • "coiueiit" text. U was 
asitieipated that finding methods to del- -et, diagnose, and correct errors 
in wrong answer" brandies would be a tine eo :#uning and uncertain process. 

In retrospeet, it appears Lliat the initial difficulties were underestimated. 
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Consul tants Used In Reviling the "Applications^ Text . The revision of 
e«ch case presentation was accomplished with the help of consultants who 
wor\cd through the cases in the following order: 

ty probers , Follow faculty members and senior roaitent physicians 
in obstetrics and gynecology were used first. These consultants, all 
of whom were qualified clinicians with experience in the application of 
"content" knowledge to the care of real patients, worked through the 
cases as subject-matter experts. Their corrections were confined for 
the most part to the "right answer" branches. Tin* purpose of their 
review was to make sure that the diagnostic and therapeutic aspects of 
the case presentations were as realistic, accurate aiul up-to-date as 
possible. After revision by several subject -nut ter •• perte, the 
"right-answer" branches of the case presentations wire i*vp<* uh to 
function rather efficiently as self-tests for the studcnL and would 
serve to demonstrate to him that he had not only mastered the "content," 
but also could apply it to reaolve a specific clinical problem. 

2 * Sophomore students . Next was a group of sophomore medical student* 

without prior exposure to either the case presentations or the "e intent" 
text# Thesa students were asked to start erch case presentation without 
having read the corresponding section in the "contenL" text and to 
work through remedial sequences in the "content" text only when specifi- 
cally instructed to do so by the text# With this approach, it was hoped 
that the sophomore students would explore as many "wrong answer 1 
branches as possible and by doing the appropriate remedial assignments 
in the "content" text, would ultimately complete the "right answer" 
branches successfully, thus demonstrating tholt learning of the material# 
It was expected that the programmer# would not only obtain data on the 







I 

I 

0 

0 

D 

D 

D 

n 

II 

o 

0 

0 



0 

0 

0 



03 - 



students' responses to the "wrong «n*wor'' frames, hut would also 
learn whether or not the remedial sequences In the "content" 
te>:t were sufficiently effective as teaching devices to permit the 
students to complete the cases successfully, 
i. Senior students. Hie third group were senior medical students who had 
completed the course In gynecologic oncology as juniors, who were familiar 
with the "content" of the linear text, but who were not experts In its 
"application." These students were asked to review the appropriate 
material in the "content" text whenever they wished, before or after 
undertaking the case presentations. It was hupul that this group of 
students would select "wrong answer" hratule . i.hieii primarily represented 
defective applications of an adequate ktiowled 5 . e, "content," rather 
than deficiencies of both "content" and "application." An evaluation of 
the senior studeat 's responses was expected to demonstrate whether or 
not these remedial sub-sequences in the "applications" text were 
adequate to permit the student to complete the case successfully. 

D* Problems of Revision - Unforeseen Difficulties, 

1* Responses of subjec t-mat tor exports . The case presentations, when 
presented to clinical subject-matter experts, benefited from much 
constructed criticism, correcting errors and deficiencies in 
the right answer branches. The wrong answer branches received very 
few criticisms. In general, the cases receive! favorable comments from 
the clinical consultants. "If these cases don't teach the students 
to work up patients, nothing will!" was tin comment oi a chief resident 
physician who had worked through all i'j ea*,. s. 
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2< Msponses of sophomore students- when the cases were presented 

to sophomore student consultants for review, it soon became apparent 
that many students were failing to complete the cases. Typically, 
a student would work through the case until he made an error which 
led him to a remedial frame, which in turn advised him to correct his 
error by working through certain frames in the "content" text. The 
student, instead of following instructions as he was advised, encouraged, 
and even paid to do, skipped to the next case where he repeated the same 
behavior pattern. It seemed that the wrong answer branches, instead of 
motivating the student to recognise his deficiencies and learn from his 
mistakes, had an opposite effect and so discouraged the student that 
he was unwilling to continue with the case even when he was paid to 
do so on an hourly basis. 

a. In itial revisions . An inspection of these wrong answer branches which 
appeared to stop the student from further learning revealed that, 
for the most part, the student's wrong choices were not treated too 
sympathetically. In some instances when the student's response had 
been outrageously inadequate or inappropriate, remedial frames 
contained remarks which could be construed as biting or sarcastic. 

As a first step in revision, the biting and sarcastic remarks were 
removed. In certain cases even the outrageously wrong options leading 
to the remarks were deleted since merely listing such an option seemed 
to be construed as a reflection on the student's competence, 
b. Effect of rev isions. The revised cases in which all student responses, 
whether right or wrong, were treated with courtesy and respect were 
then presented to a new group of sophomore students. Their reaction, 
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for the most part, was the same as that of the first group 
of students. When they recognized that they had made a 
mistake, they discontinued their efforts to manage the case or 
to learn more about the patient's disease and went on to the 
next problem. The responses (or lack of responses) by the students 

gave the programmers no clues as to why the wrong answer frames were 
falling to function. 

c. Re asons for failure .. The cau. .f the unanticipated behavior of the 
student was determined by piecing together Information obtained 

from interviews with many of these student consultants. In brief, It 
was this: 

i. The student saweach case presentation as a realistic portrayal 
of the doctor-patient relationship and recognized that the 
decisions demanded of him represented the decisions he would 
have to make as a physician caring for such a patient. He 
therefore looked upon the case presentation as a test of 
his ability to assume and carry out the responsibilities of 
a physician In a realistic manner. When he made decisions 
which were faulty and jeopardized the patient's welfare, 
or perhaps killed the patient, he saw himself as an Incompetent 
bungler in the role of p physician. 

ii. Whether the student's error was the result of a lack of know- 
ledge of the patient's disease, or from a slip-shod evalua- 
tion of the Information available to him, or from poor judgment 
in spite of having all the Information he needed to make the 
right decision, the effect upon the student was equally dis- 
couraging: he could see that he just wasn't cut out to be 

a doctor. 
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Xhese observations led to the conclusion that many students, at 
the end of their sophomore year of medical school, are 
unprepared for evidence which suggests that they, too, will make 
mistakes which must be measured in human suffering or death. For 
such students a wrong answer branch which informs them, however 
gently, that the patient has suffered at their hands is such a 
painful experience that they are unable to continue the case. 
di Resolution of problem. Fortunately for the project, certain sopho- 
more students were found who were sufficiently tough-minded to follow 
the instructions and complete the remedial assignments as required by 
the text. Appropriate corrections were made in the "applications" 
text and in the "content 11 text to permit succeeding students to 
correct their deficiencies of information and complete each case 
presentation successfully. 

3i Reaction of senior students. Senior students, when started on the case 
presentations, had a characteristic pattern of behavior: they would 

complete one or two case presentations, making serious errors, and 
then insist on being given the "content" text to review in its entirety 
before they would nroceed with further case presentations. After 
completing this "content" text, they could then complete the case 
presentations, making very little use of the wrong answer branches. 

When they did make mistakes, they were more than willing to review the 
remedial assignments in the "content" text. 

Revisions of "Applic ations" Text after the First Year** Operational Use . 

In the spring of 1965, all responses of 44 students who worked through the 
35 case presentations were tallied. A review of the responses seemed to 
justify the following observations: 
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1. There were eurpriiingly few "deed branches" In the text. Almost 
every option had been selected by one or more students. 

2. In some cases, the format, Instructions, or programming technique failed 
to communicate effectively. 

3. Students, In working through the case presentations, needed a more imme- 
diate way to compare their own performance with that of expert clini- 
cians. It was not enough, for example, simply to tell them at the 

end that they had killed the patient. 

4. In the scrambled book which made extensive use of code numbers to 
provide information, typographical errors involving code numbers or 

pagination seemed to produce Intense emotional reactions and surprising 
margin commenti. 

Prom the Information obtained from the tally of student responses, changes 
were made In less than 5% of the frames. Errors of content were also 
corrected, and a code giving the preferred choices of expert clinicians was 
added to data-gathering frames. 

P. Ord er of Approach . The tryouts of the "applications" text with student 
consultants had made It apparent that when students were given both 
programmed texts, they would use them In the following order: 

FIRST, complete the "content" text In Its entirety. 

SECOND, complete the "applications" text, working through remedial assign- 
ments In the "content" text when Instructed to do so. 

Unique Sequencing Aspects of Case Presentations . 

A * " ■ p °* e o£ 0r ig- 1 . " . * 1 Sequence , since the casas had been written In the same 
order as the "content" text, they were first tried out In this order. 
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or group of cases with students a. they progressed through the material 
rather than waiting for each student to complete the full course. 

B< — dVanta * e> ° f ° rl » tnal Op erational Use . If the sequence 

of case presentations remained In Its original order, paralleling the 

presentation of corresponding material In the "content" text, there would 
be disadvantages: 

1. The diagnosis of each case presentation would be predictable In advanc, 
and would lose Its element of challenge. The diagnostic workup 1„ 

such cases becomes a matter of drudgery without the excitement of 
dlicoverlng the unknowns 

2. Student, would recognize such a sequence a. unrealistic: patients 
with different disease, do not present themselves for care In the 
..me sequence that the!,- disease, are presented m the table of 
contents of a textbook. 

3. There could be no orderly increase In the complexity In programming 
technique, used in presenting the cases. The most demanding case, 
would appear before the student had been prepared for them by 

working through a sufficient number of relatively straight- 
forward cases. 

4. The sequence did not permit a„ orderly increase l„ the demand, made 
on the student for competence, Industry, and sophistication In dealing 
with Increasingly complex diagnostic and therapeutic problems m 
gynecologic oncology, 

gln .l _ S.qu. nee . i„ the sequence adopted for operational use, the 
case, were arranged according to the following principles: 
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1. Progressive increase In the difficulty from simple, f(UlllUr p rogr .mming 
techniques to complicated, unfamiliar ones. 

2. Progressive increase in the difficulty and co^l.xity of diagnosis 
and/or treatment* 

3. Separation of similar cases. Cases with closely related diseases, 
and cases with similar complaints but unrelated diseases, were 
separated from each other as widely as possible throughout the text, 
but were arranged in sequences according to principles 1 and 2 above. 

The result, it was hoped, was a text which for a student who msstered the 
"content" of gynecologic oncology would offer a scries of cases of progree- 
■ively increasing challenge and surprise* 

VI11 * g 2gg« f *sons o j Revisions - "Content" Text vs "Annl lotions" 

A* The “Content” Text * 

U S ^ - - t — con * ulta n te . The responses of student consultants to early 
drafts of tha content text led to extensive revisions leaving scarcely 
a frame of the original draft intact. 

2. T he _. reason . The content text require students to learn a new vocabu- 
lary and at ths same time apply it to unfamiliar contexts. The 

failures of the original draft v?re primarily failures of coimsunication. 

B * The "Applications 11 Text . 

l * ~-- ct of consultants . Tha vasponsas of student consultants led to 
very f ew changes in the format or programming techniques of the case 
presentations. The responses of the subject-matter experts (physicians 
with clinical experience) led to the correction of many defects in 

the "content" of the case presentations but few changes in the program- 
wing. 
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2. Thy »IWM . 

«. The "applications" text assumed that the student has aastercd 
th« clinical vocabulary needed for the car* of patlanta with 
gynecologic tumors. When tha atudant'a responses indicate 
that there ia a coaunlcatlon failure bacauae of hia vocabulary 
deficiencies, he ia referred to the "content" text for re Medial 
work. 

b. Tha prograa writers, after the experience of writing and re-writing 
thousands of fraaes for medical students, had bccoae relatively 
adapt at writing fraaea and preparing instructions which coeaatnlcatad 
affectively as originally written. This was fortunate, because soaa 
of the pathways in the case presentations ware never selected by 
try-out students and remained untested until the text was put into 
operational use. 
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figgBCT Mlity °f Experimental <nd control nrn..p. 

A> ~' ta SU PP° r “"« *>— "billtv Of lirnnp ColUM of . 

1963-64 . 

1. Tor * co-p.rl.cn of group. A, B, B' .nd A' by weighted grade-point 
average., see Table VII. 

2. For a comparison of group. A, B, B' and A' according to Medical College 
Admission Tost scores , set Table VIII, 

Tables giving the grade arore. and ranks of individual .tudent. are given 

in Appendix I. 

— Su PP (>r “ n « Comparability of n. o up. . t the jfedlcal roll... of 

1964-65 . 

1. For a comp.ri.on of group. A, b, B' and A' by weighted grade-poiat 
average., see Table IX. 

2. For n comp.ri.on of group. A, B, B' and A' according to Medical College 
Admission Tost scores, see Table X. 

Table, giving the grade-point average, and rank, of individual atudent. 

are given in Appendix J. 

C# fitL * . Supporting Coa e arability of Study Croun. . t oth . r 

1* Cal iforAla Colle g e of Mgdicine . The 92 .tudent. of the .opho*>r. cl... 
were divided into a control group of 45 .tudent. and an experimental 
group of 47 students on the basis of academic performance during the 
freahman year of medical achool. A.aur.nc. ha. bean given the Rea.arch 
Director of the project that this was done on aa nearly an equal basis 








TABLE VII 



COMPARISON OP EXPERIMENTAL AND CONTROL GROUPS ON WEIGHTED GRADE-POTwr 
AVERAGES FOR GROUPS OP JUNIOR STUDENTS ^AT THE MEDICAL COLLEGE OP GEORGIA 



Control Sagrtgmttl Control 

Grou P I-A II-B III-B* IV-A' 



H 


23 


24 




22 


22 


Mean 


2.42 


2.43 




2.44 


2.49 


Range 


1.75-3.19 


2.00-3. 


75 1, 


.27-3.51 


1.83-3.75 


Experimental 

$ 


N 


Mean 


SD 


^ t 


P 


46 


2.459 


•45 






Control 


45 


2.456 


.40 


.03 


HS 
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TABLE VIII 



COMPARISON OF EXPERIMENTAL AND CONTROL GROUPS ON MEDICAL COLLEGE ADMISSION TEST 
SCORES OF JUNIOR STUDENTS AT THE MEDICAL COLLEGE OF GEORGIA 

1963-64 



Group 

N - 

Verbal * 

Quantitative 
General Information 
Science 

Sum of MCATS 
Experimental 
Control 



Control Kxpe. 

I-A XX-B 



23 24 



47.0 




48.2 


50.0 




50.6 


47.5 




49.9 


51.4 




49.5 


H 


Me.n 


22 


46 


49.2 


8.5 


45 


49.5 


8.1 



iwntel Control 



III-B ' 




IV-A 1 


22 




22 


49.3 




50.0 


48.2 




51.6 


49.1 




49.3 


49.0 




49.4 




£ 




.34 


NS 





*La*t 5 dropped from all fradea 
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TABLE IX 



COMPARISON _OF ^EXPERIMENTAL AND CONTROL GROUPS ON WEIGHTED GRADE-POINT AVERAGES 
FOR GROUPS OF JUNIOR STUDENTS AT THE MEDICAL COLLEGE OF GEORGIA 

1964-65 



Group* 


Control 

i-A 




Experlnental 
II-B III-B' 


Control 

IV-A' 


N 


24 




24 


23 


21 


Mean 


2.55 




2.48 


2.46 


2.13 


Range 


2.01-3.68 




1.69-3.46 


1.66-3. 16 


1.96-3.07 




H 


ffisa 


SB 


t p 




Experlnental 


47 


2-4* 


.42 






Control 


45 


2*50 


.41 


.06 NS 
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TABLE X 






Control 



Experimental 



Control 



Group 


I-A 


II*B 


111*8* 




IV-A' 


N - 


24 


24 


23 




21 


Verbal * 


48.0 


47.8 


49.0 




46.3 


Quantitative 


48.0 


49.3 


47.3 




46.8 


General Infor«atlo n 


47.3 


47.3 


48.9 




49.4 


Science 


48.2 


48.2 


49.9 




46.5 


Sum of HCAT5 


£ 


Mean SD 


jt 


i 




Experimental 


47 


48.47 6.27 








Control 


45 


47.66 4.81 


.699 


NS 





*Last 5 dropped fro* *n grades 
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as possible and that a comparison of the Medical College Admission 
Test scores of the two groups supported the assumption that the two 
groups were comparable. The supporting information from the college's 
Registrar's Office, however, has not yet been made available to the 
project. 

2 * -- ate Univ erslty of Iowa School of Medicine. The Medical College Ad- 
mission Test scores by experimental and control groups and the grade- 
point averages by groups are shown on Table XI. 

3. The_Unlverslty of Nebr aska College of Medicine . The Medical College 
Admission Test scores by experimental and control groups are shown In 
Table XII. 

4> IbS-Pnlversltyjif North Car olina College of Medicine . The combined 
averages for Medical College Admission Test scores and grade-point 
averages for experimental and control groups are shown on Table XIII. 

5 * TheJJnlverslty of Vermont College o f Medicine. The summation of Medical 
College Admission Test scores are combined by experimental and control 
groups In Table XIV. The listing of student Individual scores l s not 
available. However, the grade-point averages for the end of the first 
and second years are given. 

D * Summary of Comparability of Study Samples . 

1. The data from the Medical. College of Georgia on previous academic 
performance. Medical College Admission Test scores, and class rank seems 
sufficient to justify the conclusion that the control and experimental 
groups were approximately equal. The experimental design of the 
project permitted the distribution of the students to be biased favoring 
the control groups, but the data suggests that this bias was not large 
enough to impair the comparability of the groups. 

2. The data from the other medical colleges participating in the second year 
of the project are less complete than that from the Medical College of 
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TABLE XI 

COMPARISON OF EXPERIMENTAL AND CONTROL GROUPS ON MEAT SCORES AND 
GRADE-POINT AVERAGES FOR JUNIOR STUDENTS AT THE 
STATE UNIVERSITY OF IOWA SCHOOL OF MEDICINE 
1964-65 



GROUP 


N 


V 


Q 


GI 




Sci 


GPA 


Control I 


10 


53.2 


55.8 


55.4 




55.4 


3.1 


If 


13 


50.3 


55.3 


54.2 




52.8 


3.2 


V 


11 


52.0 


57.7 


57.1 




55.6 


3.4 


VII 


11 


52.4 


56.0 


52.4 




55.5 


3.0 


Experimental 
















II 


9 


53.0 


51.0 


55.8 




52.9 


2.9 


III 


10 


50.5 


59.1 


57.3 




56.9 


3.0 


VI 


13 


53.2 


55.4 


54.7 




53.5 


3.1 


VIII 


11 


48.3 


50.0 


52.9 




52.6 


2.8 


Sum of MCATS 




N 


Mean 


T * 


P 






Experimental 




43 


53.6 


243 


NS 






Control 




45 


54.2 









CODE 

V - Verbal 
Q * Quantitative 
GI * General Information 
Sci * Science 
GPA * Grade-Point Average 
-MCAT-» Medical College A dmlaalon TV»«f 



*Wilcoxon T rank teat 







-78 



TABU XII 



COMPARISON OF EXPERIMENTAL AND CONTROL GROUPS MCAT SCORES 
OP JUNIOR STUDENTS AT THE 
UNIVERSITY OF NEBRASKA COLLEGE OF WDICINB 
1964-65 



Control I 


10 


47.6 




52.2 


Si 

48.4 


Scl 

49.3 


III 


9 


49.6 


52.5 


52.6 


54.3 


V 


9 


56.2 


51.0 


57.7 


54.7 


VII 


10 


55.4 


53*4 


58.3 


53.2 



Experimental 



II 


ii 


50.4 


53.0 


51.9 


52.2 


IV 


10 


50.4 


52.0 


53.0 


51.7 


VI 


11 


51.4 


54.1 


54.9 


52.0 


VIII 


10 


48.3 


51.7 


52.9 


49.6 


Sum of MCATS 




£ 


Mmp 


1* 


£ 


Experimental 




42 


52.9 






Control 




38 


51.9 


229 


NS 



CODE * 

V - Verbal 
Q - Quantitative 
GI « General ZnforMtlon 
Eel « Science 

MEAT " Medical College Admission Tests 



*Wllcoxon T rank test. 



0 



a 



0 



» 
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TABLE XIII 









GRADE-POINT AVERAGES FOR SENIOR STUDENTS aTtHE 
UNIVERSITY OF NORTH CAROLINA COLLEGE OF MEDICINE 
1964-65 



““*** AIW 



GROUP 


N 


V 


Q 


GI 


Sci 




Experimental I 


14 












III 


16 


52.0 


51.8 


51.3 


51.6 


2.0 


Control n 


14 


59.8 


56.3 


49.2 


51.8 


2.2 


Sum of MCATS 


N 


Mean t* p 




Experimental 


30 


51.7 








Control 


14 


51.8 


16 NS 





n 

J 



0 

0 

0 

a 

D 



CODE ' — — 

V - Verbal 

Q ■ Quantitative 

®I* General Information 

Scl “ Science 

GPA ■ Grade-Point Average 

MCAT ■ Medical Collegi Admission Tests 



♦Wllcoxon T rank test. 









.1 



I 

I 
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TABLE XIV 



i 



COMPARISO N O F EXPERIMENTAL AND CONTROL GROUPS ON TOTAL SUMMARY OF MCAT SCORES 
AND FIRST ‘ AND SECOND ’ YEAR GRADE-POINT AVERAGES FOR jS STUDENTS 
AT THE UNIVERSITY OF VERMONT COLLEGE OF MEDICINE 
1964-65 



0 



GROUPS 




N 


AVERAGE OF 
MCAT SCORES 


GRADE-POINT AVERAGE 


pi 




FIRST YEAR 
AVERAGE 


SECOND YEAR 
AVERAGE 


y 


1 Control A I f 


Ilia 


22 


56.7 


83.5 


n v MiyvuEf 

82.7 




Experimental B 


II , Illb 


23 


56.7 


81.1 


82.3 


o 


! 


P-value 




MS 


NS 


1 

NS 


u 



0 

MCAT ■ Medical College Adaleelon TeaCe 

Q 

D 

Q 

0 

Q 

0 
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D 

0 

0 

D 






Goorgia. Each of those schools, however, had . long-.t.nding 
administrative policy of requiring that the dlvlilon of each clan 
Into smaller group, bo accomplished without di.tortlng the 
academic comparability of the groups, in no school were the 
students permitted to assign themselves to group, by a process of 
■elf -selection. There 1. no evidence that there ws. . deliberate bias 
In the assignment of student., favoring either the control or the 
experimental groups, In any school participating In the project. 

“* ~ Cont <-"t Comparab ility fo r Control and Experimental » 

A- Comparability of Content at the m llese of gaorgU, 1963-64 . 

1. Da t a available . The Information presented In Chapter Two, page 9 
describing the teaching materials, and In Chapter Three, page 16 
de.crlblng the experimental design of the project, and In Chapter Four, 
page 35, describing the development of the teaching materials suggests 
that In the first year of the project every effort was -ad. to Insure 
a virtual duplication of coverage of "content" In control and 
experimental groups. Material, are available to make detailed compari- 
sons of the coverage of content in the eight hour, of tape- 
recorded lecture, with the 30,000 word, of programmed "content" 

text. Such a study, although possible, is not practical within the 
liMlta of thi« report# 

2 * ----*** lrrH,>vanc,p of content co mparability . The results of such a 
study would still leave room for large errors in interpreting the 
result, of the testing program. The 216 queation. of the Special National 
Board Examination, of OB-GYN Neoplasms sampled less than 10% of the 
material presented by the lecturer and by the programed text and 
Included about a doren questions which were not covered in either 



D 



« 
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teaching method. Since the content covered by the tests represented 
such a small portion of the content covered by the two teaching programs, 
slight changes of emphasis in the teaching of content might be expected 
to have produced important changes in test scores, without producing 
obvious discrepancies between the two teaching methods in the coverage 
of content. The reason that this did not occur is that neither the 
lecturer nor the program writer, at the time of preparing teaching 
materials, was aware of the precise content of the questions of the 
Special National Board Examinations in OB-GYN Neoplasms, nor did either 
make any effort gb a later date to include in his leaching "anewers" 
to the examination questions. 

B. Cojsp arab 1 11 ty_o f Content at the Medical College of Georgia, 1964-b5 . 

1. During the second year of the project, all students in control and 

experimental groups received the same ''content** programmed text. Thus 
the basic coverage of content was the same for all groups. The experl* 
mental groups, in addition, received the "applications" text. This 
text was designed to contain as little new ''content" as possible. Th e 
control groups received no cosparable special training in "application" 
other than their regular work in the wards, clinics and conference 
rooms, which the experimental groups also received. In this year the 
comparability of course content between control and experimental 
group* depended not so much on the nature of programmed text* they 
received ae on their distribution of atudy time between the texts. 

C. Separability of Course Content In Other Medical Schools, 1964-bS. 

1. IxperlmfntMi groups . In the five schools participating in the cross- 
validation study, experimental group* received uniform treatment; they 
were given the "content" programmed text at the beginning of the period 





o£ Instruction. The experimental groups „ere denied access to the 
programs of formal classroom instruction given to some control groups. 

2. Control g roups . There was no uniformity of "content" or method of 

teaching control groups i„ the different schoolc. Each school was free 
to teach ns much or as little of the "content" of gynecologic oncology 
as it saw fit. Instructors of control groups were not denied access 
to the Special National Board Examinations of OB-GVN Neoplasms adminis- 
tered to their students as pre- and post-tests, whether any instruc- 
tors took advantage of this opportunity to alter their teaching of 
control groups to teach "answers" to the examination questions is 
unknown, but lu considered to be most uulikcly t 
_T_lwe. to Criterion Records # 

A. C o nee t ion of Data . AH students participating in all phaaca of the project 
in all achoola were required to turn in completed time recorda atating 
their study time in learning gynecologic oncology (Sec Appendix F). 

The recorda required information in the following cutogoriea: 

1. iloura attending lccturea or seminars in gynecologic oncology. 

2. Hours reading about gynecologic neoplasms in assigned textbooks. 

3. Hours reading other texts or articles about gynecologic neoplasms. 

4. Number of patients with gynecologic neoplasms assigned to student. 

k* 'K P* t J ent * (previously unassigned to a student), 
b. "Old" patients (reassigned). 

5. Hours studying programmed text (if any). 

Students turned them in at the conclusion oi the clerkship or period of 
instruction. Students were advised that their records would be mors accurats 
and hence more valuable to the project if they were kept on a daily or 
weakly baaia rather than fiilad in only at the end of the clerkahip. 
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Students were also advised that they ware to record only the 

they spent learning gynecologic oncolosy and should exclude 

fro» the record any time spent studying other aspects of obstetrics 
And gynecology. 

*• jSSBB of Study Time Kecordi , (gee Teble XV) 

1 ‘ Mg-^cal CoUw of Caort ia, 1963-64 . Hie experimental groups achieved 
their learning with a saving of time almost equivalent to the time that 
control groups spent attending the lectures. 

2 * Si georala. 1964-65 . Experimental groups completed 

both the "content" taxt and the "applications" text in about the iih 
tiaa that the control group spent completing the "content" text alone. 

Tor both groups, the expenditure of tine was slightly greater than 
that of thu control group of the preceding year. 

3 * £tva. other schools,, 1964-65. The study time of experimental groups varied 
considerably fro* school to school, the study time of control groups 
varied even more greatly and followed no consistent pattern with regard 
to tha experimental groups at each school. The data, horever, do not 
favor a saving of tlaie for experimental groups. 

IV* laming of "Contents" 

A * E*>»d*eto Achievement of teaming of "content." 

*• i?e41cal Collage of Geor gia, 1M3-64 , The achievement of experimental 
groups was at laaat as good as that of the control groups and was 
accomplished with a significant saving of tima (sea 'fable XVI). , 

2 * M*-- SSllttt ot G worsts. 1964-65 . Experimental groups who received 
both tha "content" text and tha "applications" text learned "content" 

■s efficiently as did control groups who received the "content" text 
a Iona (sea Table XVXX). 



TABU XV 



SUMMARY OF TIME RECORDS, ALL SAMPLES 
CONVENTIONAL INSTRUCTION, CONTROL GROUPS 




M. C.Ga. 
Calif. 
Iowa 
Nebr. 

N. C. 

Vt. 



1963- 64 45 

1964- 65 45 
1964-65 45 
1964-65 39 
1964-65 15 
1964-65 23 



TEXT COMPLETION TIhF 



Aver 

14.7 

13.0 

15.5 

12.2 

13.6 

18.7 
18.9 



Range 

9-26 

5-24 

8-25 

5-21 

4-27 

4-57 

7-34 



S.D. 

4,0 

4.5 
4.4 

3.6 
6.3 

12.6 

6.6 



7.0 0-17 

9.4 0-20 

28.6 0-70 
4.6 0-30 
20.3 0-96 
4.6 0-25 
11.5 0-48 



4.4 
5.2 
18.0 

7.9 

25.4 
7.6 

12.5 



21.7 
22.4 
44.1 

16.8 

33.9 
23.3 

31.9 



13-38 

8-35 

12-88 

5-42 

4-108 

8-82 

10-79 



M.C.Ua. 1964-65 45 
•'content * 1 text 
"application 1 * text 



22.9 

14.2 

8.7 



13-35 

8-29 

4-18 



6.3 

4.4 

3.4 



1* Formal lectures 

3.‘ ^°2 n hou^tu^r C ^SnJ U hi^ > d , * n ?. , *' ,lor * «v«ry two v«ek>. 

$■ »*«*"•*». varying in °" Clerk,hlp Wlth 0B eM « 5hail - 

5. Lecture* in Junior year (clarkahip in a.Sor year). 



5.9 

6.8 

18.7 
9.0 

27.1 

15.8 
16.5 



M.C.Ga. 1964-65 43 

PROGRAMMED INSTRUCTION, "COMPOSITE" TEXT GROUPS 

11.2 0-50 ll.l | 34.1 17.53 139 



TABLE XVI 



COMPARISON OF EXPERIMENTAL AMD CONTROL 
EXAMINATIONS IN OB-GYN NEOPLASMS, 

1963-64. 



GROUPS ON SPECIAL NATIONAL 
MEDICAL COLLEGE OP GEORGIA 



BOARD 



WB PariMtloni 



Control 




Pra- 




fiii! 1 


A and A* 


45 


57. S 


85.0 


27.2 


Experimental 










B and B 1 


46 


56.3 


85.9 


29.6 


t *v 




• 8 


.6 




Significance 




NS 


NS 





*Diff*r»nc* in miq scores. 






I 
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IABLE XVII 



0 



D 

D 

D 

D 

a 

0 

0 



"“iarrazs ssb »»» 



n 




gra- 


goat- 


££ln 


U Control 










A and A' 

M 


45 


51.6 


83.3 


31,7 


1 | Experimental 










B and 8' 

mm 


47 


53.7 


84.7 


31.0 


D 




1.0 


.7 




Slgnlficftncc 




MS 


MS 





♦Difference in Man acoraa. 
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3* Five other med ical college!, 1964-65 , In every school, the "content** 
text consistently produced more learning than did the school's conven- 
tional teaching program, (see Table XVIII). 

B. Retention of Learning of Content . 

1* Method . The Examination in Obstetrics and Gynecology of the National 
Board of Medical Examiners, Part II, was administered to whole classes 
at the Medical College of Georgia according to the following schedule: 
May 1963, the Junior Class 

May 1964, the Junior Class and the Senior Class 

May 1965, the Junior Class and the Sertior Class 

a. The class tested in May 1963 as juniors and again in May 1964 as 
seniors was a control group. These students had completed their 
training in obstetrics and gynecology as juniors prior to the 
start of the project and had no contact with the project as seniors 
during the year 1963-64. 

b. The class tested as juniors in May 1964 and again as seniors in May 

1965 was an experimental group in that all students in this class 

P*^^ c ^P®fc®d in the project, half as controls in the lecture group, 
and half as experimental students in the programmed text group. 

c. The class tested as juniors in May 1965 was an experimental group, 
since all students participated in the project, half receiving both 
the "content** text and the "applications" text and the other half 
receiving the "content** text alone. A measure of retention of 
learning for this group will not be available until after they 

are re-tested as seniors in May 1966. 

Special data collected. The National Board of Medical Examiners supplied 
categorical analyses for each class and for each experimental and control 
division within the classes which participated in the project. 

Category 11, Neoplasms, represents the "content" of the project's 
teaching programs. The results are shown in Tables XIX - XXI. 
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TABLE XVIII 



COMPARISON OF EXPiTOIMENTAL AND CONTROL GROUPS ON SPECIAL NATIONAL BOAin 

examinations in ob-gyn neopiashs. five oiHn^SK&oSr 

1964-65. 



NB Special Exanlnat Iona 









Poit- 


Gain 


U. of Vermont: Cont. 

Medical School Exp. 


22 

23 


56.8 

57.6 


80.2 

85.2 


23.4 

27.6 


t 

Significance 




.03 

NS 


2.3 

<.05 


U. of N.C. Cont. 

Medical School Exp. 

t* 

Significance 


11 

30 


67.9 

61.4 

2.9 

<.01 


84.9 

89.3 

2.1 

<05 


17.0 

27.9 


U. of Nebraaka Cont. 

Medical School Exp. 


38 

42 


61.6 

64.6 


79.8 

84.7 


18.2 

20.1 


t 

Significance 




1.3 

NS 


2.7 

<01 


State U. of cont. 

Iowa Medical Exp. 

School t 


45 

43 


63.1 

61.9 

i 


88.6 

93.3 


25.5 

31.4 


Significance 




. l 

NS 


3.2 

<.01 




California cont. 
College of Exp. 
Medicine t 


45 

47 


45.5 

47.0 


76.0 

81.0 
* o 


30.4 

34.0 


Significance 




1. A 

NS 


<01 










TABLE XIX 



SCORE MEANS AND STANDARD DEVIATIONS FOR EIGHT GROUPS OF 
JUNIOR STUDENTS ON NATIONAL BOARD PART II COMPREHENSIVE EXAMINATIONS IN 
OBSTETRICS AND GYNECOLOGY, MEDICAL COLLEGE OF GEORGIA, 

MAY 1964 and MAY 1965 



1963- 64 

A - Control 
A'- Control 
B - Experimental 
B f - Experimental 

1964- 65 

A - Control 
A'- Control 
B - Experimental 
B v - Experimental 



N 


NBflE 

Final Gradf 




iHay 1964) 




Moan 


22 


23 


79.1 


5.1 


22 


83.4 


4.6 


24 


79.2 


4.2 


22 


78.9 


4.4 



(Hay 1965) 

24 78.8 5.4 
21 82.5 3.5 
24 82.2 4.9 
23 



0 

0 




a 

o 



80.6 



5.2 



CZ3 




ft 




n 



u 



a 



n» 

Li 

n 

u 



n 



a 

n 

u 



n 

U 




D 

D 
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TABLE XX 



COMPARISON OP MEAN PERCENTAGES, BY CATEGORIES, OP NATIONAL BOARD CANDIDATES 
tm «!LIOR STUDENTS TAXING COMPREHENSIVE EXAmSaSn CANW “ TBS 
IN OBSTETRICS AND GYNECOLOGY, MAY 1963, MAY 1964, MAY 1965 



CATEGORY 


Juniors 1963 


MEAN PKRCINTAq 
Juniors 1964 


SS 

.Tnninra 1 QLC 


- (Examination Coda) 


(KMB) 


ilmo 








NB 


MCG 


NB 


MCG 


NB MCC 


1. 


Embryology, Anstomy snd Physiology 
of the Female Organs of Repro- 
duction 


67.3 


63.3 


72.7 


65.5 


77.8 


66.8 


2. 


Physiology and Ecology of Woman 


74.3 


65.9 


70.0 


65.9 


75.8 


73.9 


3. 


Normal Pregnsncy; Physiology, 
Biochemistry, Psychology, Diagno- 
sis! Mansgement 


1 

65.3 


62.0 


63.1 


54.2 


77.0 


66.7 


4. 


Physiology and Conduct of Norjnal 
Labor and Parturition; the 
Newborn 


64.9 


63.8 


65.0 


57.4 


71.0 


63.0 


5. 


Ihe Puerperiura; Normal and 
Abnormal 


66.2 


58.7 


77.1 


64.0 


76.0 


48.0 


6. 


Complications of Pregnsncy 


71.9 


67.2 


69.7 


64.8 


66.7 


57.1 


7. 


Complications of Labor snd 
Delivery 


58.3 


56.6 


68.8 


57.9 


75.9 


69.0 


8. 


Disturbances of Function 


63.2 


60.4 


84.7 


76.4 


75.3 


66.8 


9. 


Anatomic Pelvic Disorders 


65.4 


60.6 


50.0 


42.6 


83.6 


80.0 


10. 


Infections 


78.5 


71.9 


68.9 


56.8 


52.6 


44.3 


n. 


Neoplasms 


71.9 


62.8 


66.2 


69.9 


67.9 


73.7 



NB " l*cen«ure? Ver * 8e me,n percent,8e * for Mn*W»te» taking the National Boarda for 



MCG ■ Medical Collage of Georgia aean percentage avaragea taking the National Boarda. 




BY CATEGORY, FOB MEDICAL OOVUCS OF CCOKTTa 

‘KM SS2.5S S3 S3S =£* 



umm 




Juolor Y«tr 




Senior Year 




&) 


fiartmT 


(Slot) 


Control 


Cxperim 


1 


72.7 


64.5 


66.5 


77.8 


53.8 


63.6 


2 


70.0 


67.1 


64.7 


75.8 


61.1 


63.2 


3 


63.1 


34.7 


5 3.7 


77.0 


61.6 < 


61.4 


4 


65.0 


59.8 


55.1 


71.0 


63.5 


61.7 


3 


77.1 


64.9 


63.1 


76.0 


27.0 


44.0 


6 


69.7 


67.7 


62.1 


66.7 


54.1 


52.4 


7 


66.6 


62.0 


53.9 


75.9 


63.4 


61.5 


• 


04.7 


75.5 


77.3 


75.3 


67.8 


63.5 


9 


30.0 


42.6 


42.6 


83.6 


77.4 


77.9 


10 


60.9 


59.6 


54.7 


52.6 


43.3 


38.1 


->11 




-Z2.A, 


JE2i2- 


-4ZaL 


-22idu 


-#lL 




D 

D 

D 



n 

u 



u 

n 

i 

U 

n 

U 

n 

u 

n 

; 1 

U 



u 

n 

u 

n 



0 



-93- 



3 * Sugary of re.ult. of » rctcntlnn'» .w, 

* A. measured by the National Board Examination., the experimental 
teaching Pt.gr..., both b, by t „ t , ptMue . d 

• remarkable improvement in average .core, for Category 11, 
Neoplaama, for atudent. participating in the project, «. chared 
with atudent. who had received their in.truction in the y., r prior 
to the .tart of the project. The improvement in Category 11 W a. 
not accompanied by improvement, in the other ten categories 
b. The results of re-te.ting a year later indicate that the interval 
of one year i. sufficient to aboli.h evidence for improved .core, in 
Category 11 . Re-te.ting of senior, at the time of their graduation 
revealed c.eutially the ..me pattern of .core, whether their 
teaching as Junior, had been by means of programmed text., .pedal 
lecture, prepared for the project, or by the ordinary departmental 
teaching conducted prior to the start of the project, 
c. «.e programmed text deigned to teach "content - dio not produce 
•n improvement in retention of learning when retention wa. 
measured over a one-year interval duri„ ;; which the students were 

prevented from relearning from the text by luing denied access to 
it. 

learning o f '■Application.-' 

A * il£ "Ann lieat ion 11 „ y oral 

1. Hdhod. At J u ‘h.-Uai College of Georgia, In January and in Hay of 
each year o| the project, two visiting Judges, working a. a team, 
conducted interviews at 30-minute interval, with all student, serving 
clerkships in obstetrics and gynecology during the current and the 
preceding quarter. Students from experimental and control gs.up. 



D 




were presented to the judges in scrambled order. IV Judges conducted 
the Interviews using u case presentation format in which students had 
an opportunity to demonstrate their ability to apply their knowledge 
of the ''content” of gynecologic oncology to specific problems of 
patient care. 

S c o rin g, Each judge rated each atudent independently with regard to his 
knowledge of "content” and his akill in its "application.” The Judges 
then discussed the student's performance with each other and arrived 
at a "final pooled grade" by consensus. All ratings were expressed 
using the following five-point scale. 



0 » Unsatisfactory, without rcaervatlon. 

1 - Borderline or questionable knowledge, performance, 

ability. Passing subject to Promotion Board. 

2 ■ Satisfactory knowledge and performance - average. 

3 » Better than average 

4 - All-round .super lor student with above average prepara- 

tion, capabilities, performance, Interest, knowledge. 

drive, etc. 

JSiliilts* The average acorea for the combined control and experimental 
groupa of each yea.' of the project at the Hedical College of Georgia 
are shown in Table XXIX. » 

Summary of results . 

a. At the Medical College of Georgia in 1963-64 there was a positive 
but not significant difference in che combined scores of control 
groups, taught by lectures, and experimental groups taught by the 
"content" programmed text favoring the experimental groupa. 

b. At the Medical College of Coorgia in 1964-65, there was an observ- 
able difference in the combined scores of experimental groups, 



Q 



* 

l 

i 



TABUS XXII 



— =bt- - ssias skm. 



D 

D 

D 

G 

0 

0 

D 

D 

0 

0 



DATE 




GROUP 


H 


m 


AVERAGE 


January 13-17, 


1964 


Control A 
Experimental B 


23 

24 


46 

62 


2.0 

2.6 




May 4-8, 1964 




Experimental B' 


22 


3ft 










Control A' 


22 


46 


1.7 

2*1 




January 11-15, 


1965 


Control A 
Experimental B 


24 

24 


42 

56 


1.6 

2.3 




Hay 3-7, 1965 




Experimental B' 


23 


52 

45 


2.3 

2.1 








Control A* 


21 




1963-64 




Control 


N 

45 


tks" SD 

2.0222 ,866 


t 


* 






Experimental 


46 


2.1739 .957 


.789 


HS 


1964-65 




Control 


45 


1.9333 1.044 










Experimental 


47 


2.2978 1.021 


1.736 
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who received the "composite" text (the "content" end the "appli- 
cation*" text used together) and the control group* which received 
the "content" text alone. The difference favored the experlaental 
group and wee eignificant at the 10* level of confidence. 

B * ^Wf. ugtll * n t_of "Appll c»i.ion" by Written Examination* . 

1. Th._di agnostic teat*, wine tab-ite* clinical problem-solving teete 
were prepared for uee in the project at the Medical College of Georgia 
in 1964-65. Of these, eight were primarily dlagnoatlc in nature, ihe 
foraat and content of theee teste have been deecrlbed in Chapter II and 
in Appendix E. Ihe echedule for adMlnleterlng the teat* le given in 
Chapter III, page 31. Aw students' performance at the beginning of 
their clinical training in obetctrlce and gynecology wae eo erratic and 
resembled eo closely the scores that might be obtained by non-medical 
persons marking the answer sheets by chance chat a detailed evaluation 
of their pre-ueet performance wae abandoned as being unworthy of further 
effort* 



The ecoring of the post-tests, however can be considered under the 
three skills Measured by the tests* 

*• SiiMSSSSiSm ftocsss (comprehensivensss and appropriateness of 
diagnostic workup). Four post-tests had sections dealing with 
History, Physical Examination, and Diagnostic Tests and Procedures. 
In effect, each student was given over 350 "yes-no" options in 
gathering information about the four cases* fliese options had 
bean categorized by the residents as: 

i. Essential, required by nature of the problem, 

ii. Routins screening or survey item, 

iii. Useless but harmless item, 
iv* Contraindicated, harmful item. 
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Tables XXIII and XXIV compare the performance of control and experi- 
mental groups in selecting options In these categories. 

The data In the tables suggest that experimental groups who had 
received the "applications" text were more thorough In the 
diagnostic workup of their patients than were students In the 
control groups who received only the "content" text. The experi- 
mental students, In general, selected more "essential" and more 
"routine" items than did students In the control groups. In a less 
consistent fashion they also tended to select more "useless" or 
contraindicated" Items than did control students, especially in 
the first half of the year. The results are not directly comparable, 
however, because the series of tests given to experimental and control 
groups differed markedly In the number and classification of "useless" 
and "contraindicated" items In each test. 

b * — ~K nost * c product (accuracy and comp 1 <>*:_ ness of diagnosis). 

U Unrecognized communicati on problem, in each of the four 
"diagnosis" post-tests, the Diagnostic Product sections 
recorded satisfactorily the diagnostic judgments of the 
criterion group of senior resident physicians, and were 
considered acceptable by faculty members who took the test. 

With the Junior students, however, this section was an 
unqualified failure, for reasons which wore not suspected 
until too late In the academic year to make appropriate 
corrections. Ihi difficulty lay In a failure of communication 
of th » test format. 

11# J-^ qt:rilct:lon8 oo The diagnostic product section of 

oath test consisted of a list of as many as 50 different 
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TABLE XXIII 



COMPARISON OF FIRST EXPERIMENTAL AND CONTROL GROUPS ON CLTNTfiAT prort 
SOLVING TESTS, HINTON NEDICH, SIOOENIS NEoSTjoK^^S 

1964-65 



l 



i 



Control Group A N ■ 24 



# 

Choices 



Z 



Possible Significance* 

Choices Level 



L ■ Routine Items 
M - Indicated Items 




1729 

1595 


42 

62 


4152 

2592 




Experimental Group R 


Total 
N - 24 


3324 


49 


6747 


^05 


L ■ Routine Items 
M - Indicated Items 


2322 

1991 


59 

72 


3960 

2784 




Total 


4313 


64 


6744 




Control Group A N i 


- 24 










N ■ Not indicated 
P ■ Contraindicated 




157 

36 


11 

19 


1440 

192 




Experimental Group p 


Total 
N - 24 


193 


12 


1632 


NS 


N ■ Not indicated 
P ■ Contraindicated 




274 

83 


20 

29 


1368 

288 






Total 


357 


22 


1656 





a 

a 

a 

n 

o 

o 

o 

0 

0 

0 

0 





TABLE XXIV 



COM 5J? ISOS 0F SEC0ND EXPERIMENTAL AND CONTROL GROUPS ON CLINICAL PROM pm 
SOLVING TESTS, JUNIOR MEDICAL STUDENTS MEDIcTSlLEGe S ^GM 

1964-65 



Experimental Group N * 23 

L « Routint Item 
M - Indicated Items 

Total 

Control Group A* N - 21 

L ■ Routine Items 
M - Indicated Items 

Total 

Experimental B 1 H - 23 

N - Hot indicated 
P * Contraindicated 

Total 

Control Group A* N « 21 

N * Not indicated 
P ■ Contraindicated 

Total 



* 

Choices 


% 


Possible 

Choices 


3129 


66 


4738 


1982 


75 


2645 


5111 


69 


7383 


2148 


52 


4158 


1800 


70 


2583 


3948 


59 


6741 



288 


21 


1403 


43 


23 


184 


331 


21 


1587 


195 


16 


1197 


47 


25 


189 


242 


17 


1386 



Significance 

Level 




HS 






disease states. The student was asked to select AS MANY 
as he considered appropriate to define the patient's problem 
completely. 

iii# Fer^forinance by students. Students whose previous clinical 
experience was entirely in surgical disciplines recorded 
their diagnosis in this section by selecting only the "mosr 
likely," "best established," or "working" diagnosis which 
they could use as the basis for planning further management 
of the patient. Departmental residents and faculty members 
in obstetrics and gynecology also followed this approach. 

This behavior represented the appropriate response called 
for in the instructions. Unfortunately, students whose 
prior clinical experience had included non-surgical specialties 
such as internal medicine and psychiatry followed a different 
pattern. They selected not only the "working" diagnosis, but 
also mos_t of the other diagnoses, many of which were mutually 
exclusive, which they had considered as diagnostic 
possibilities during parts of the workup. 

Explanation of failure. The students who adopted this 
approach seemed to be following a recommended medical 
practice: To list as a diagnostic possibility every diag~ 

nosis, no matter how unlikely, that cannot be excluded 
with absolute certainty. Unfortunately, the test instruc- 
tions, by failing to make it clear to the student that 
he should specify only the "best" or "working" diagnosis, and 
not a list of differential diagnoses, made it impossible 
to distinguis\ the students who had selected a list of 
differential diagnoses as a matter of prudent medical 
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practice from the other students who had arrived 
at the correct diagnosis only after making a number of bad 
guesses and recording them, and furthermore made it 
impossible to identify still other students who had 
attempted to arrive at the right answer by using a "shotgun" 
approach to the answer sheet on the assumption that there 
could be no penalties for wrong answers, 
v. Sc oring abandoned . The failure of the test to communicate 
to the student instructions which were taken for granted 
by the departmental residents, faculty members, and trial 
students used in developing the tests made it necessary to 
abandon attempts to score the students' performance in 
Diagnostic Product. 
c * Therapeutic product. 

i. F ormat and scoring . Ea ch of the four post-tests emphasiz- 
■'! ing diagnostic skills also contained a therapy section. 

These sections contained as many as 50 items from which as 
many management options as necessary could be selected. 

A system of pattern scoring had been developed by the 
criterion group of resident physicians. This scoring system 
worked satisfactorily for faculty members. 

“* — U ' dent P cr ^ or mance. I„ the four tests, the scores of stu- 

dents in the therapeutic product sections were extremely 
erratic and seemed to bear little relationship to the 
students' performance in other parts of the test, 
iii- Scoring dilemma. Since in each test the formulation of 

an appropriate program of therapy required that the student 
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first have in mind the correct diagnosis, only those 
student's who had the correct diagnosis in mind at the 
start of the section on therapy could have their skills 
as therapists measured on an equal basis. Students who 
started the section with a wrong diagnosis in mind 
would invariably select a plan of therapy which was inappro- 
priate for the patient (though not necessarily for their 
erroneous diagnosis). The problems of communication in 
the diagnostic product section of each test were such that 
in the majority of the tests, it was impossible to determine 
whether or not the student had the correct diagnosis in mind, 
iv. Scoring abandoned * The lack of this information made it 

impossible to score the Therapeutic Product sections of the 
tests completed by different groups of students on an 
equal basis. Therefore attempts to score these sections 
of the four tests were abandoned as futile. 

2. Management test . 

a . Scoring . Of the clinical problem-solving tests, only one, Test B , 
was developed with the specific purpose of measuring skill in 
management of a complex problem. The diagnostic process In this 
test was a simple one and Included only eight Items. The test, 
described In detail in Appeudlx E, required the student to make as 
many as eight difficult therapeutic decisions, In sequence, each of 
which involved the weighing of multiple factors and variables. In 
each sequenced decision, one variable at a time was altered and the 
student was asked to make one selection from as many as ten options, 
most of which remained constant from decision to decision. The 
therapy section of this test was scored using the trial scoring 
system. The categories are given as follows: 
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A 

B 



F 

2 

2 Z, 



■^■^•peutic Product (treatmen t and dis pQ .fnnn\ 

Beit management (recommended at our institution) 

A jsassr t <o£ "" — — « othtr 

A ^n table raana8 * ment ("V involve more riak or mutilation 
iSEHom!? Mry bUt lB *PP ro P rl * t « to the problem) 
conaequenc“r 8enent ’ (U,U,Uy Undertreatmen t, with non-fatal 

™j.T» ££ SJ51S5{i2S! 1 ’“ gr ‘"‘ ■*"» « 

^tgrass&.%r r ^ » f «— *- « ««—««> 

n n n » _ _ 



Z A, Z B, 2 C, 



Z D, z F Subsidiary item Implying that the 

**|^*® r ip^ letter must also be ecored 
whether or not the atudent marked it 
* x*’ Z A iB ^rked, acore both A and 
Z A> 



In acorlng thla test, these categorlee were weighted as follow. : 

A * 100 E * 25 

B * 100 p m o 



Reiult^ The .cores of the four group, of students, two control 
and two experimental, participating In the project at the Medical 
College of Georgia during 1964-65 are given In Table XXV. 

3. Summ ary of result ., students In experimental group, who received both 
the -application- and the -content- programmed text, achieved ob.ervably 
higher average .cores In thl. -management- te.t than did the control 
student, who received only the -content- programmed text. The difference 

favoring che experimental group Is significant at the 17 . level of 
confidence. 

Attitudes of Students . 

A * M fjitude Toward -Content- Text . Table XXVI .how. a tally of the .even tot.! 
classes given the attitude .urvey on sample Items: 3, 4, 7, 19, an d 21. 

For a full summary of these attitude .urvey., ae . Appendix 0. Sample, of 

statements made by .tudent. on the open-ended Item, on the attitude .urvey 
•re given in Table XXVII. 
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TABLE XXV 



COMPARISON OF FOUR GROUPS TAKING MANAGEMENT TEST. TEST B 
MEDICAL COLLEGE OF GEORGIA 1964-65 * 



AVERAGE SCORER 
TEST B 









N 


(MAXIMUM POSK^BU; 7(\n\ 


% 


Group 


A 


Control 


24 


219 


31 


Group 


B 


Experimental 


24 


481 


69 


Group 


B' 


Experimental 


23 


536 


77 


Group 


A' 


Control 


21 


450 


64 



N 

Control 45 

Experimental 47 



Mean 


SD 


, t 


p 


326 


184 






508 


94 


5.93 


^01 
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TABLE XXVI 

sample of attitude survey items - "content 11 text 

ITEMS; 3, 4, 7, 19, and 21 (Seven samples total N - 323) 

3. Now that I have completed the course. I think this form of programmed teaching 



1. 

2 . 

3. 

4. 

5. 



Excellent 
Very good 
All right 
Poor 

Completely unacceptable 



68 

153 

56 

14 

3 



4. Compared to MOST teaching method. I have encountered, this form of induction 



1. 

O 

• 

3. 

4. 

5. 



Far superior 
Better 

About the same 
Inferior 

Extremely inferior 



44 

177 

41 

30 

1 



7. The programmed material was intellectually challenging 



1. Throughout 

2. Most of the time 

3. Sometimes 

4. Infrequently 

5. Very rarely 



20 

150 

100 

21 

5 



19. For iomc other medical gubjectu I would choose programmed Instruction 



1. 


In preference to all 
other ferma 


63 


2. 


With approval 


109 


3 • 


Maybe 


38 


4. 


With hesitation 


18 


5. 


Only as last resort 


9 



tixt were available in a bookstore, I would recommend that others 





i. 


Buy it and use it 




H 




frequently 


150 


2. 


Buy it and use it 




i i 

L J 




occasionally 


94 


Wi 


3. 


Borrow but don 9 1 buy it 


38 




4. 


Accept it as a gift only 


11 


n 


5. 


Avoid it completely 


4 


J 


••Number of stude 


*nts making this choice 






f 

> 



TABLE XXVIX 



SAMPLE OF STATEMENTS MADE BY STUDENTS ON OPEN-ENDED ITEMS ON 
THE ATTITUDE SURVEY 
(Six Schools) 



A. 



Things you liked shout this programmed course: (five most common comments) 

I# Brevity, conciseness, clarity 
2# Easy learning 
3« Illustrations 

4, Organization 

5. Challenging and Interesting 



Bs 



Things you disliked. about this programmed course: (five most cownon comments) 



1 . 

2 . 

3. 

4 . 

5. 



Format, i,Ca, lack of references 

Too long or too snort, i*e#, subject not covered or covered 
Ambiguity 

In some areas, I felt, too often, I was "spoon fed, 11 
Don t feel that subject matter is retained 



too much 



C. 



Changes you think should be mads to improve the programmed text: (five most 
common comments) 



Is Add references, index, tabic of contents 
2s Kora illustrations 
3« Casa presentations 

4. Correction of typographical and grammatical errors 
5s Eliminate or add repetition 

D. How dr you think programs such as this could be of most value to nwH^ai 

(five most common cowaents) — ■ ■cuuuici* 

3a Review, i,e*, National Board tests 
2a Mass of basic information 
3s Continuing education 

4s Programmed courses should be available to students and practicing 
physicians for review of post-graduate education, 

5. Should be instituted in other fields of medicine. 



X?i P re ?S n “"« Information have you found valuable for learning 

while In medical school? (flva most common consents) 



1> Experience, 1 . e. , conferences and rounds 

2. Text books 

3. Lantern slide lectures 

4. Nona batter than programmed learning 

5. Casa Presentations 



H ‘ oier 50 h Mn”med) ,ubJec,:, that mlfht be P^asmed. (10 most often listed out of 



Biochemistry 


6. 


Physiology 


7. 


Microbiology 


8. 


Pathology 


9. 


Pharmacology 


10. 



Mtdlclna 

Pediatrics 

Cardiology 

Endocrinology 

Obstetrics 
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B ‘ —■ t - tUde Tow " rd " A PP lic,tio n»" Text. A cample of the attitude, toward che 
"application.'' text are ahown in Table XXVIII. For a complete aurvey aummary 
Appendix Pa 

C ‘ aigga ?f At - titud< ‘ Surv «Y» - The reaction of nearly all atudenta toward, 
both programmed texta waa atrongly favorabla. Negative comment, were in- 
frequent, demonatrated no conaiatent trend, and ware often contradictory. 
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IABLE XXVIII 

SAMPLE OF ATTITUDE SURVEY ITEMS - "APPLICATIONS" TEXT 
“• 4> 5 > (Ml ■ - 42) 

'• C0 " P ‘ t ' <1 “ tl,e Pto S ra», I find the cun pte.ent.Mon. to be 



1* Completely Acceptable 

2. Acceptable 

3. All right 

4. Unacceptable 

5. Completely unacceptable 



15 

22 

4 

0 

0 



4 ’ 5n e be hln8 me £ ° apply my kn0Wledgc t0 P«tlent. thl. progr.m, I expect. 



1. Most valuable 

2. Valuable 

3. All right 

4. Of little help 

5. Of no help 



11 

25 

3 

0 

0 



5 ’ other*: ^ • v,1Uble ln * book. tore, I would r.con»cnd that 



1» Buy It and use It frequently 

2. Buy It and use It occasionally 

3. Borrow but don*t buy It 

4. Accept It as a gift only 

5. Avoid It completely 



^Number of students making this choice. 



16 

20 

4 

0 

0 
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CHAPTER SIX 
INTERPRETATION OF DATA 



1 ' gg-^-V.ancc of Teaching Goals to Measurements . 

Measurements of the effectiveness of a teaching method arc meaningful only 
to the extent that the goals or objectives of the methods encompass the 
goals or objectives of the testing methods. If, for example, the 
teaching methods in this project were devoted chiefly to the diagnosis, eval- 
uation, and treatment of carcinoma of the cervix, and the testing methods 
emphasized instead, the differential diagnosis of ovarian tumors, to the 
neglect of carcinoma of the cervix, it could be anticipated that the data 
from the testing methods would indicate that the teaching methods were 
relatively ineffective. An accurate description of the results of such a 
testing program would be that the teaching program, devoted to the diagnosis 
and treatment of carcinoma of the cervix, was relatively ineffective in teaching 
the differential diagnosis of ovarian tumors, and that the effectiveness of 
the teaching of carcinoma of the cervix was not adequately appraised. 

in many phases of this project, there were discrepancies between the teaching 
goals and the testing goals. These discrepancies, varying in kind and degree, 
are important in interpreting the results of the various testing programs as 
measurements of teaching effectiveness. 

A. E n .Bl i . eit Goals of Teaming Methods . The "Requirements of Course, 1 ' 

(Appendix H) an outline of the content of the teaching program of the 
project, was prepared by the faculty of the Department of Obstetrics and 
Gynecology in an attempt to express its teaching goals in behavioral terms, 
lltc goals were expressed, however, without regard to the restrictions imposed 
by time, by the limited availability of facilities and patients, and 
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personal limitations o£ the students and the faculty. The "Requlrementa 
of Cour.e" deacrlbe. . level of knowledge which 1. approached, but may not 
be achieved, by realdent physicians In obstetrics and gynecology after 
montha or years of specialized training (at leaat three yeara beyond the 
junior atudent level). Although exceptionally gifted medical atudenta 
could poaalbly reach auch levela of competence In a few weeka of clinical 
clerkahlp, the explicitly atated goala of the "Requlrementa" are unrealiatlc 
aa repreaentatlona of the content of the formal teaching and teatlng programa 
for the average atudent. 

B * -- " h - erent Shortcomings of Teaching Methods . All three of the teaching methods 
used in this project (lectures, "content" programmed text, and "composite" 
programmed text) were Intended to meet the teaching goals of the "Require- 
ments of Course." For none of these methods was it possible to measure 
directly the extent to which it achieved the atated teaching goals. The 
shortcomings of each method were related to the limitations of instructional 
time and to the unsuitability of the teaching medium to the subject matter. 

1. The lectures . The lectures attempted to present in a didactic manner 
the "content" of the "Requirements of Course." They emphasized the 
importance of the correlation and "application" of this content to the 
problems of caring for individual patients. The lecturer left the student 
to do his own correlating and applying of this information in patient 
care on the we^ds and in the clinics* 

2 * ■ Th --- " content:l1 tgx £ « This text had essentially the same teaching goals 
as did the lecture method. Each method of teaching was unable to give 
the atudent supervised experience in the application of "content" infor- 
mstion to specific problems of patient care* 
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3. a s J^ogosl^ text . The "application" text (used i„ conjunction 
with the "content" text) attempted to do equal Ju.tlce to both the 
"content" and "application" aspect! of the ( oal B expressed In the 
"Requirement, of Course." The t-t, unlike th. "requirements." was written 
with strict regard to the time available to student, for studying this 
subject. This time limitation required that the number, variety, and 
length of cases be restricted to the extent that studentc were not given 
adequate drill to achieve the degree of competence specified in the 
"Requirements of Course. ,f 

C* Implied Go t It of Criterion Measures . 

1# ■ fof - yritten ^content" Examinations i n OB-OYN Neoplasms of eh, Hatlonal 
Bo a rd, of Hedlccl Examiner. . An inspection of the item, in these 
special National Board Examinations in OB-GYN Neoplasms had confirmed 
that most of the questions are for "content" information of the kind which 
characteristically is presented to students in didactic lectures or in 
standard textbooks of gynecology. The special "content" examinations 
represented nearly .11 of the National Board*, pool of question, in 
obstetrical and gynecological neoplasms. The 216 items were distributed 
into Tests A and C and Tests B and D as follows: 

Cateiory 

Ovary and Tube 
Sarcomas of uterus 
Fibroids of uterus 
Endometriosis 
Endometrium 
Endometrium and Cervix 
Cervix 
Vagina-Vulva 
Choriocarcinoma 

Ihe teaching materials for the project were developed with no attempt to 
match, category for category, the content or the omphasis of questions in 



Number of Items 



For mi A & c 


Forms B & D 


30 


25 


4 


3 


6 


5 


17 


13 


12 


10 


10 


12 


13 


27 


11 


8 




5 


100 


108 
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the special National Board Examinations in OB-GYN Neoplasms. In fact, 
one category represented on the examinations, choriocarcinoma, was 
deliberately excluded. A comparison of the "content” of the examinations, 
question by question, with the content of the linear text, frame by frame, 
indicates that the examinations covered about 107. of the material 
presented in the text, with many disparities of emphasis. Since even 
the text fell far short of the goals of the "Requirements of Course," it 
is apparent that the gap between what was tested for by the examinations 
and the explicit teaching goals of the faculty was a very wide gap indeed. 
The shortcomings of the lecture series and the "content" text and the 
shortcomings of the special National Board Examinations in OB-GYN Neoplasms 
were well-matched in kind if not in decree. The examinations may be 
considered as appropriate measures of the effectiveness of these two 
teaching methods, provided on** accepts these limited goals. 

2 4 Th - oral examinations. The purpose of the oral examinations, to measure 
the "application" of content knowledge to specific problems of patient 
care, is referred to in the "Requirements of Course" only by implication. 
The judges of the oral examinations had to decide for themselves, without 
objective guidelines, not only what student responses represented 
appropriate management" of the patient, but also whether the responses, 
appropriate or inappropriate, were to be classified as "content" or 
application." The judges could dihigree over the appropriateness of the 
behavior the student exhibited, as well as over its classification. 
Fortunately, the examiners shared similar viewpoints as experienced 
teachers trained in the same specialty, and were in apparent agreement 
most of cite time, but what they agreed on was never specified. 
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Si£ written problem-solv ing te»t» designed to test '’application. 11 
The«e tests were detailed examples of "application" and demanded from 
the student specific skills which were not defined In the "Requirements 
of Course." They called for a knowledge of "content," and a degree of 
aklll In Its application that exceeded the capabilities of some resident 
physicians and faculty members, and moat students. The teats Identified 
competency In expert performance, but were defective In Identifying the 
kinds and degrees of incompetency exhibited In most other perfor manc es. In 
measuring a few specific skills, the testa exceeded the explicit demands 
of the "Requirements of Course." 

4. Summary of criterion goals . 

a. The special National Board Examinations in OB-GYN Neoplasms were 
appropriate criterion measures of the teaching of "content" as defined 
In thla project. They were inadequate, however, in their representa- 
tion of the teaching goals of the project as set forth by the faculty 
in the "Requiremente of Course." 

b. The methods used to teach and 'eat for aklll in "application" repre- 
sented a much closer approach to the teaching goals of the "Require- 
ments of Course" than did the "content" methods. Unfortunately, the 
testing methods for "application" (the oral examinations and the 
clinical problem-solving testa), contained procedural flaws which 
limited their value as measuring devices. 

Subjec tive Estimates of 

A * Faculty Opinions of Teaching Methods . 

1. The_ lecture aeries. There was general agreement among faculty numbers 
who monitored parts of the lecture aeries or listened to tapes recorded 
during the series that the lectures were of outstanding quality with 
appropriate and well-used visual aids. 
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2. The programmed texts . 

»* The "content" text, No questionnaire was submitted to the faculty 
members in the departments of obstetrics and gynecology using 
this text, but verbal comments have been received from faculty 
members in most of the schools using the text. In general, they 
injected the text and found it unimpressive. It is described as 
skimpy in the coverage of subject matter, not always accurate, often 
confusing, excessive in the demands it makes of students, dull, repe- 
titious, and requiring an unnecessary amount of writing. In their 
opinion, most textbooks would be more readable, lively, and 
authoritative. 

b. The "composite* 1 text. Faculty members did not recognize the 

"composite* 1 text as such. Most of them could complete all cases 
in the "applications" text without resorting to the "content" text 
at all. Faculty members who reviewed the "applications" text 
seemed to consider it a significant advance in student teacning. 

They recognized it as a text which give* the student experience in 
responsible decision making of a nature that is otherwise not avail- 
able to him as an undergraduate medical student. 

3* Testing ncthodfr . Faculty members who subscribed, even in a general way, 
to the principles set forth in the "Requirements of Course" were apt 
to dismiss the data from testing using the special National Hoard Examina- 
tions in OB-GYN Neoplasms with remarks such as, "The experimental plan 
may be sound, and the results of the examinations may be valid, but the 
examinations do not test what we reall> try to teach." There was a 
recognition that ttjeir own testing tended to remain "content-oriented," 
and that there was difficulty in interpreting the results of examinations 
with an "application" orientation. Even the oral examinations in this 
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project tended to begin with case presentations to test "application" 
and end with conventional questions on "content." Many faculty members 

believed that only in real-lire situations could one teach or test fer 
’'application. 11 

B. Student Opinions . 

1. T he lectures . With regard to the lectures, student opinions were sharply 
divided. The majority opinion was that the lecture, were well-organised, 
well-delivered, and presented an excellent coverage of subject matter. 

The minority opinion was that the lectures were "worthless" because 

almost everything the lecturer said could be learned from reading a textbook. 

2. The programmed__te -|ts. There seemed to be a consensus among the student, 
in the project that tne introduction of. the programmed rsxts was an 
Improvement in the teaching program which would facilitate learning. 

There was a minority opinion that conventional methods of teaching 
permitted more efficient use of the students* time. 

c * Evaluation of Subjective Opinion! . 

1. Faculty opinions . It would appear that <;he adverse Judgment of the 
faculty members with regard to the "content" text is basea upon their 
observation that it falls .hort of their teaching geala rather than 
on their estimate of it. effectiveness in attaining more limited goals. 

Their favorable opinion of the ease presentation text appears to be 
based on it. face validity. The cases seem to teach many of the goals 
which are not covered by other educational media, including the "content" 
programmed text. But with regard to teaching effectiveness, it has been 
well established in other studies that subject-nutter experts, whether 
or not they are also programming experts, are poor Judges of the teaching 
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effectiveness of programmed material* in their own Held* of competence. 
The opinions in thi* study are probably no exception. 

2. Student opinions . The student opinion* as to the teaching effectiveness 
of the programmed texts have remained favorable in spite of a generally 
negative faculty attitude. The student opinions, however, are valuable 
only as expression* of attitude. 

III. Objective Estimates of Teaching Effectiveness . 

A. Lectures versus "Content” Text . 

1. Learning of "content," immediate achievement. 

*• Experimental plan. Throughout the project, a classic pre-test- 
post“test pattern was followed. In only one of the six schools 
participating in the project was there a significant difference 
betwesn experimental and control groups in their pre-test scores. 

In this one school (The University of North Carolina) the difference 
favored the control group. In the six schools participating in the 
study, only one had a known bias in its division of students into 
control and experimental groups. Thi* one school (The Medical 
College of Georgia) biased its samples on the basis of previous 
academic performance in favor of control group*. This bias, however, 
did not appear In the pre-test scores. Probably in all the schools 
there were occasional differences in treatment between experimental 
and control groups which were not called for in the experimental 
design of the project. None of these, however, have been appare t. 
b. Tests . The measure of effectiveness of immediate achievement of 
learning of "content" was the gain score of the post-test over the 
pre-test scores on the special examinations in gynecologic neoplasms 
of the National Board of Medical Examiners. 
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The«e test! had high reliability. Each question In theo. examination 
had been ahown to be a conalatent performer In nationwide teetlng. 
There waa a remarkable uniformity of reeulte In the mean pre-teat 
and poet-teat acoree of the varloue experimental and control groupe 
in the project# 

Theee tests had good face validity. There seems little question that 
the tests were adequate measures of portions of the body of 
knowledge taught in this project. 

c. Evaluation, for the five medical schools in which the "content" 
programmed text was compared with the schools* conventional method 
of teaching* the immediate achievement of learning for those in the 
experimental groups was significantly higher than that of students 
in the control groups as measured by the special National Board 
Examinations in OB-GYM Neoplasms. There were no exceptions to this 
pattern. At the Medical College of Georgia* where control groups 
were given a specially prepared series of lectures of the highest 
quality possible* there was no significant difference between 
experimental and control groups in the immediate achievement of 
learning of "content." The data in this project justify the con- 
clusion that the "content" text has a teaching effectiveness for 
Junior medical students which is usually superior to conventional 
classroom instruction 

2 * "**j , .«ntlon 11 of learni ng of "confnt. 11 Thi. wa. mta.ur.d only at th. 

Hadlcal Coll.g. of Caorgla. Th. maa.ur. of r.t.ntlon wa. th. dlff.r.nc. 

In th. icor.a in Cat.gory 11 (N.opla.m.) of compr.hcn.lv. examination, in 
Obit. trie, and Gynacology of th. National Board of Medical Examiner a, 

Mi adminl.t.r.d to atudtnt. at th. and of th.ir junior y.ar and again 
at th. .nd of th.ir aanlor y.ar. 
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«• SE S - rimenf 1 plin . In this phase ot the project, the experimental 
plan, in retrospect, was clearly defective, 
i. The students who took the tests as juniors in May 1964 were 
divided into control and experimental groups. The control 
groups who received lectures in the first and fourth quarters 
of the academic year had lecture notes to study right up to 
the time of the examination. Experimental students, who had no 
lecture notes, had been deprived of their progranwed texts from 
9 to 18 weeks previously. The collection of the programmed 
texts from the students was uecessary to prevent the texts from 
falling into the hands of control students. But it also 
deprived experimental students of written materials to study 
prior to the final examination. Whether experimental students 
made use of control students' lecture notes io unknown. Never- 
thcless, the experimental plan at the end of the junior year 
in 1964 was unbalanced, and compared retention over a 9 to 18 
week interval for the experimental, students against relearning 
from lecture notes (after an Interval of from 1 to 27 weeks 
from the lectures themselves) for the control students, 
ii. In May 1965, when another version of the tear was administered 
to the same students as graduating seniors, the imbalance in 
the experimental design persisted. Experimental students 
had no written materials other than standard textbooks to 
study. Control groups had their lecture notes still available 
to them. Although all students were advised not to study for 
this examination, it war not determined whether or not they 
heeded thii advice, 
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Summary. The experimental plan for measuring retention VAf9 
defective in several ways: 

(*). There was a failure to recognize that control students 
learned and relearned from lecture notes as well as 
from the lectures themselves. 

(b). There was a failure to rcccgnizc that experimental 

students who had completed the programmed text could re- 
learn rapidly from their own text or could borrow a copy 
if they were sufficiently motivated. 

(<*)• There was a failure to recognize that students who had 
completed the programmed text differed from control 
students who had not used the text in their motivation 
for this review from the programmed text, 

(d), For three-fourths of each class (all hut the last quarter), 
the immediate 1 post-test at the end of the academic year 
was in itself a measure of retention ( and rr learning ) 
administered 9 to 27 weeks qftcr completion of the course 
of instruction, Probably, the most rapid decline in 
proficiency occurred during the first nine-week interval. 

The delayed post- test, administered a year later* measured 
a more gradual decline, and this decline was influenced 
by uncontrolled relearning by both control and experimental 
groups. 

(e). It was unfortunate that the operational plan of the 

projact required the collection of the programmed texts 
from the students at the end of the instructional period. 

In ordinary medical school courses, it would be unthink- 
able for a teacher to confiscate textbooks, workbooks, 
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or lecture notes from his students! In retrospect, ell 
students should have been permitted to retain their learning 
materials, whether programmed or not, and the objective of 
measuring "retention" should have been replaced by a more 
realistic one: measuring "retention - plus relearning," 

b. Tests, In each comprehensive National Board Examination In Obstetrics 
and Gynecology, Part II, the neoplasma category comprised about 

40 questions. In the measurement of retention, the performance of the 
control groups A and A' was compared with the performance of the 
experimental groups B and B' using the same questions. For the four 
groups of over 90 students, 40 questions probably represented an 
adequate sample of "content" knowledge. 

c. Summary . 

i. The "content" programed f»vt produced a "retention" of 

learning which was slightly superior to that of conventional 
instruction. 

11. "Retention," as measured in this study, Included uncontrolled 
and unmeasured opportunities for relearning, 
ill. The potential of the programmed text, once completed by the 

student, to facilitate rapid relearning was not recognised early 
enough in the project to be studied or evaluated. 

B * Text versus "Content" Text . The "composite" text (consisting of 

the "applications" text used in conjunction with the "content" text) whs 
compared with the "content" text used alone in studies conducted at 
Medical College of Georgia in the second year of the project. Measurements 
ware made of the Immediate achievement of learning of "content" and immediate 
achievement of learning of "application." 
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X. Achievement of ltarnim of "contsnt." 



*• Kxpegjmentel plan. In this phase of the project , the ssne expert* 
mental plan end the same special National Board Examinations in 0B»GYN 
Neoplasms were used to compart the effectiveness of the "composite" 
text with the "content" text as had been used in the previous year 
to coiqpare the "content" text with conventional teaching, 
b. Evaluation, xhe test results indicate that the "composite" text 
was equally effective in teaching "content" as the "content" text 
ws when used alone. Since the composite text was not designed to 
improve the teaching of "content" over that of the "content" text 
alone 9 the results of fVs phase of the testing were the anticipated 
ones. 

2* Achievement of l earning of "application," The central purpose of the 
entire research project was to measure the effectiveness of the "com- 
posite" text in teaching the "application" of "content" knowledge to 
lndlvl' ual problems of patient care, fltis was the crucial issue in the 
project, compared with which all the other successes and failures were 
of secondary importance. Measurements of effectiveness of learning of 
"application" were by oral examinations, and written teits. 
a. Oral Examinations . 

i* Experimental, plan for oral examination. . Examination, were 
conducted twice yearly, during a weak in January anu a weak 
in Hay. During 'ach examining week, atudent. aerving in tha 
current and the preceding cl.rkehipe in obatctrica and gyne- 
cology were preeented to the examlnera in acraablad order, 
for aoma atudanta. the examination rapreaenteJ a meaeure of 
immediate achievement o f learning; for othera, the examination 
repreaent.d a meaeure of retention of learning after an eight-week 
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Interval during which time the student had devoted his energies 
to learning another discipline (general surgery). The ABB 'A' 
sequence of the clerkships was designed to produce a balanced 
experiment. At the first examination session In January, 
retention of learning of students in the control group after 
an eight-week interval was compared against the Immediate achieve- 
ment of learning of the students In the experimental group. In 
the May examination, the relationships were reversed and reten- 
tion of learning of students In the experimental group was 
compared to Inmedlate learning of students in the control group. 

In the second year of the project, this pattern was repeated. 

11 * — lt3 ln the desl B n - There were two defects in the experimental 

design, both of which favored the control groups. 

Sampling b ias favoring controls . In each year of the 
project, students of borderline academic status had been 
excluded from the second control group serving at the 
end of the year. 

Treatment bias. Students ln control groups ln the first 
year of the project had lecture notes which they could 
review at any time up to the examination. It may be 
assumed that for the first control group the oral exam- 
inations were a measure not just of retention over an 
eight-week period but also of relearning from lecture 
notes. In contrast, their counterpart experimental group, 
examined In the latter half of the year, had been deprived 
of their programmed texts eight weeks before the examina- 
tion. Since few of these students had notes to review, 
the examination represented "retention" without relearning. 
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In the second year of the project, all students received [ 

the "content" text at the beginning of the clerkship. ’ 

Students In experimental groups received the "applications" 
text In addition. Students In the control group serving 
In the first quarter of the academic year turned In their 
"content" programmed texts a week before the oral examina- 
tions. Students In the other control group, serving In 
the last quarter, turned In their programmed texts after 
the oral examinations, at the time of the final written 
examination at the end of the academic year. Experimental 
students serving In the second quarter had been deprived 
of the programmed texts at the time of the oral examinations. 
Experimental students serving In the third quarter had been 
deprived of the "applications" text at the end of the 
quarter, but turned In the "content" texts much later, at 
the end of the academic year. At the conclusion of the 
final written examination, students In the experimental 
group, all of whom had completed the clerkship nine weeks 
previously, were asked to what extent they had reviewed or 
studied the "content" text In the Intervening nine weeks 
In preparation for either the oral examinations or the 
written final. This Information Is shown In the first 
column of Table XXIX. 

In every Instance the students did their relearning after 
the oral examinations. In preparation for the written 
final examination. Thus, the treatment bias, permitting 
students to retain their "content" texts can be assumed to 
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IABLE XXIX 

1964-65 





HOURS STUDY 
TO RELEARN 
AFTER 
CLERKSHIP 


SCORES ON 
NBME FART II 
MAY 1965 


SCORES ON * 
POST TEST 
TAKEN AT END 
OF CLERKSHIP 


SCORES ON 
ORAL EXAMS 
TAKEN AT END 
OF CT.XPKSHTP 


Experimental B 1 










SUMMATION 


30.3 


1353 


1917 


52 


N 


23 


23 


23 


23 


AVERAGE 


1.3 


80.6 


83.3 


2.25 


RANGE 


0-4 


72-91 


64-97 


0-4 


Experimental B 










SUMMATION 


No Data 
(Books offi- 


1972 


2063 


56 


N 

AVERAGE 


cially un- 
available to 


24 


24 


24 


students) 


82.2 

75-96 


86.0 


2.33 


RANGE 




69-101 


1-4 



*Speclal National Board Examination in OB-GYN Naoplaama 

*52 53"i“ ' 1Z2£ SSfSSs ,?*“ — - •— » — 



I 

D 

9 

9 

9 



S 



9 



9 



ERIC 










• 125 - 



hav* had no affect on tha oral axaalnation scores. 

Similar Information from thair countarpart group in tha 
balanced daalgn, tha firat control group, la laaa collate, 
but interviews with aoma atudanta from thla group suggest 
that they mada llttla or no affort to raviaw borrowed 
copiaa of the text in preparation for tha oral examination. . 
lil * V of oral examinations . lha experimental daalgn of 

tha project did not permit direct maaauramanta of the reliability 
of oral examlnationa. *a-h student received only one half-hour 
examination in "applications" of "gynecologic oncology" by the 
team of visiting examinera. Within a weak or two of this 
examination, however, each student was subjected to other oral 
examinations by teams of faculty members of the department of 
obstetrics and gynecology of the Medical College of Georgia. 

Those ex am in ations dealt with other phases of obstetrics and 
gynecology and the format and scoring system were similar to that 
used by the visiting examiners, xha oral examinations, both 
"project" and "departmental," followed a similar pattern. The 
two members of each examining team tended to agree rather closely 
in their evaluation of a given student's performance in an oral 
examination. Different teams evaluating different performances 
by this student, however, disagreed to an amazing degree. Indi- 
vidual student scores on all oral examinations in obstetrics and 
gynecology by viaitlng examiners and by departmental examiners 
are shown in Appendix Q. They tend to confirm the notorious 
variation among oral examinations in measuring the competence 
of individual students. They leave unsettled the question of 
whether the evaluation of a single examining team can be con- 
sidered reliable or not. 
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lv# Validity of oral laminations (or ,r Hhat was tested?' 5 ). ^ 

review of taped Interview* suggests that the oral examinations 
landed to conform to a pattern: 

(a). The student vould be presented with a problem case. 

He would be asked either to request further information 
or else to proceed to outline his plan of management up 
to the point where he needed further information. A dia- 
logue would then proceed with exchanges of information 
between the examiners and the student. As long as the 
topic under discussion was the management of a particular 
patient, the interview could be considered as an appro- 
priate test of Application." 

<b). If the student failed this test of "application" r n d 
mismanaged the patient, he would be presented with a 
new case presentation and the exchange of information 
between the examiners and the student would again take 
place. After each mismanagement, he would be presented 
with new or altered case presentations, and the interview 
would continue in this fashion until the end of the examin- 
ing period. 

(c). If the student passed the test and managed the patient to 
the examiners 1 satisfaction,, he might be given additional 
problems to manage by exchange of information with his 
examiners. Often, however, the examiners, once satisfied 
that this student was adspt at "application" and that 
hs could develop appropriate managements for paiients 
other students mishandled, turned from evaluation 
of "application" to evaluation of the student. Questions 
tended to be related to "content" and often were to see 
how widely the student had read in the literature of gyne** 
cologic oncology. 

v ‘ Interpretati on of results of oral examinations . The oral 
examinations as measures of "application" were contaminated 
to varying degress with "content" testing. For come students, 
the oral examination was very nearly a pure test of "appll* 
cation." For the majority, the amination was a mixture, in 
varying degress, of "content" and "aoplieation" testing. There 
was a tendency for students who performed well in "application" 
to be subjected to more stringent "content" testing than 
students who performed poorly in "application." This tendency 







of the explaining team to ahlft toward ''content" tasting 
when faced with a student who ahowed superior skill In "appli- 
cation" made It unlikely that the superior skill In "application" 
would survive to be represented In the student's "final pooled 
grade" unless the student also had a superior knowledge of 
"content." 

The evaluation of the judges was tantaaount to a judgacnt of 
the student's general competence in gynecologic oncology rather 
than a test of skill In "application" alone. 

The ratings were made on a 5-point scale, but since the extremes 
of this scale were seldom used, the oral examinations in 
effect divided students Into three categories: 

(■)• The less competent. 

(b) . The competent. 

(c) . The more than competent. 

As has been shown, the division of students into control and 
experimental groups was made on the basis of previous academic 
performance, with a bias favoring control groups. A testing 
procedure which tended to measure general competence rather than 
specific skills would be unlikely to disclose striking 
differences between the groups being compared. Furthermore, it 
has been shown that the achievement of learning of "content" 
for all experimental and control groups at the Medical College 
of Georgia in both years of the project was essentially the 
same. Differences In skill In "application" between control and 
experimental groups might well have existed, and yet have been 
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»o camouflaged by similarities In knowledge of "content" 
between groups that they would not show up in ths oral 
examination grades. 

vl * fealMtton of oral examination xfca fact that In the first 
year of the project there was no significant difference In 
the oral examination grades between control and experimental 
groups, neither of idiom had been taught "application" speci- 
fically, was to be expected, in view of the relative insensl- 
tlveness of the oral examinations In measuring skill In "appli- 
cation," It Is possible that the observed difference In the 
■econd year of the project, significant only at the 10* level, 

under-representa the difference In "application" skills. 
bl Written teste. 

*' Pi*" for VrlfttB HlflL* Ibe experimental plan for 

administering the clinical problem-solving tests was similar 
to that of the special National Board Examinations In OB-CYN 
WtONlaems. it called for pre-tests at the beginning of each 
clerkship and post-tests at the end. Barts of this testing pro- 
gram were discarded for reasons which have already been discussed. 
IHe "pre-tests," for example, although planned as measures of 
the student's entering repertory of "application" skills, were 
eo far beyond the student's level of competence that they had to 
be abandoned as measuring devices and were retained only to 
familiarise the student with the format of the tests. 



“* wsamai Uausuite hrffftr m four of the five 

post-testa only the "diagnostic process" sections were scored 
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The»c diagnostic sections, taken togsther, comprised nearly 
400 Items of clinical information which students could select 
or bypass. In these sections of the tests, the performance of 
students in control and experimental groups was compared to the 
performance of a criterion group of senior resident physicians 
in obstetrics and gynecology. The data for these four teats 
are consistent: The experimental groups came closer than 
did the control groups to emulating the performance of the 
criterion group; the experimental group was consistently more 
thorough; it selected more "routine" or screening Items, more 
"indicated" or essential items than did the control group. 

Taking all the "routine" and "indicated" Items together, the 
difference In Diagnostic Process performance of the experimental 
groups is significant at the 5% level of confidence. This 
difference can be reasonably explained as an effect of the "com- 
poaite" text on students in the experimental group in teaching 
them increased thoroughness in their diagnostic workups. 

There was no significant difference between experimental and 
control groups in their avoidance of useless and contraindicated 
diagnostic items. Thie result was to be expected as the case 
presentations in the "applications" text did not attempt to 
identify for the student which diagnostic item were "useless" 
or "contraindicated." in the summer of 1965, after the 
completion of the etudy, revisions were made in the "applicationa" 
text to help the student recognise the inappropriate diagnostic 
item in each case. 

W r J - t tcn.po»t-t»sts: Therapeutic Producy . 0 f the five clinical 
problem-solving post-tests, only one was designed as an extended 
test of the student's therapeutic Judgment in selecting 
solutions for a series of complex clinical problems. 
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Thl* test (Problem B) was scored with the help of criterion 
groups of senior resident physicians In obstetrics and gynecologv. 
In this test, the difference In nean scores between the experi- 
mental and the control groups favored the experimental groups 
and was significant at the 17, level of confidence. Again, It 
sesma reasonable to attribute this Improved performance In 
patient management to an effect of the "applications" text In 
teaching the experimental group improved skills in the applica- 
tion of "content" knowledge. 
iV. Summary of "application" testing. 

(a). The oral examinations suggest, but fall to prove beyond 
the 107, level of confidence, that the "composite text" 
was more effective than the "content" text in teaching 
■kill In the "application" of content knowledge to indi- 
vidual problems of patient care. Defects In the experi- 
mental design of the project and <n the examinations 
themselves Introduced a number of uncontrolled and un- 
measured variables. In general', however, the defects 
tended to minimise or obscure differences in "application" 
■kills. 

(b). The wiitten examinations were limited in scope and 
measured specific "application" skills. In the 
specific problems covered by the tests the data indicate 
that experimental atudents demonstrated significantly 
greater skills in patient management, both in the collec- 
tion of diagnostic information und in the formulation of 
a plan of therapy, than did the control students. 
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IV. Studio* of Teaching Efficiency # 

A. Time M a Factor . The efficiency of a taaching Method variea inversely 

with the study time required for students to reach a given level of criterion 
performance. Estimates of the efficiency of a teaching method are no more 
valid than the time-to-crlterlon records upon which they are based. 

B. Validity of Time Records. The following observations, made at the Medical 
College of Georgia, reflect upon the validity of Lhese records. 

1. A minority of students attempted to turn in totally blank time records 

at the end of the course. These incomplete forms were returned immediately 
to the students for completion and invariably were handed in complete, 
in every detail, a few minutes later. Such records represented no 
more than retrospective guesswork, and were certainly less accurate than 
records which other students maintained on a daily basis. The last-minute 
records, however, could nef be distinguished from similar-appearing 
records which were handed in coveted, but may have been filled in only 
a few minutes or days earlier, rather than throughout the clerkship. 

2. A minority of students, especially those in control groups, failed to 

restrict their recording of itudy time to the subject matter of gynecologic 
oncology. They included in their record all the time they spent reading, 
attending conferences, lacLUrcr, and seminars in all topics during the 
clerkship, and thus produced study-time figures as much as ten times 
greater than the average for their group. Such records could be revised 
only by the erring students themselves at a later date. Some of these 
students seamed never to have grasped the meaning of "oncology" or 
"neoplasms" or "tumors" and insisted on including in their records many 
hours spent studying irrelevant subjects, and even their revisions per- 
petuated curricular Impossibilities. For example, some students listed 
many more hours attending lectures in gynecologic oncology than were 
available to tlnra during their clerkship. 
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3. The control groups’ record, of the tine .pent reading In gynecologic 
oncology io«etliM. Included tiM .pent re-reading the imne Material. 
Student. In experlMntal group, kept a fraM-by-fraM record of their 
hour, .pent working through the prograsmwd text the flr.t tlw, but kept 
no .y.teMtlc record of hour, .pent reviewing or relearning from the 
text. Thu. with regard to relearning or review tlM, the control and 
experlMntal group.’ record, were not directly cowparable. 

4. Certain student, who failed to kaap objective record, during the clerkship 
May have arrived at th.lr final figure, not by Independent e.tlMate. 
of their own effort but by collusion with .lMllarly delinquent colleague.. 

C * gLoble— with T1 m Kecor d. at Other Medical School.. Nearly all student. In 
the other Mdlcal .chool. participating In the project turned In completed 
tlM records, the data from which were analyzed at the Medical College of 
Georgia. Although no observation, were received which would reflect upon 
the correctness of the data obtained In other schools, one May a.suM that 
procedural problftM were encountered .lMllar to those at the Medical College 
of Georgia. For example, bom student, recorded that they had attended Mny 
More iioura of classes In gynecologic oncology than were offered to theM at 
their school. 

D * A gJ agtngnt of Raw Data froa Time Record. . Every effort was Mde not to tanper 
with the tlM record, of students, it we. felt that the students’ record, 
should be accepted at thalr face value and that the correction of "error." 
could only be acco^ll.hed at the risk of Increasing the bias of the data. 

In the end, however, the following correction, were Mde uniformly on all tlM 
records • 

1. For control group., the mbxImum hour, attending lecture, or seminar. In 
gynecologic oncology was set at the hour, they actually attended the 
lectures, if a roll call was taken, or at the hours stated up to the mxImuih 
number of hours of Instruction available to them, when no roll call was 
taken. 
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2. Arithmetical error, were corrected. It wa. ...umed that when a di.crep- 
ancy existed between a student's final total, and hi. day-to-day or 
week-by-week record., the latter were more reliable and hi. arithmetic 
should be corrected accordingly. 

3. Student, at the Medical College of Georgia in both control and experimental 
group, whose time estimate, were considered extraordinarily high were 
asked to make sure that their estimate, were restricted to the field of 
gynecologic oncology t and to make such correction, in their record, a. 
they considered appropriate. If they made no correction., their recoru. 
were accepted nchanged. 

B * SUSSSSL Efficiency Data . At the Medical College of Georgia, in both 

year, of the project, control and experimental group, reached approximately the 
•ame level of performance on the special National Board Examination, in OB-GYN 
Neoplasm, learning under condition, which permitted direct comparison, of 
atudy times, m the five other Medical College., experimental group, reached 
a higher level of criterion performance under condition, which did not permit 
a direct compari.on of .tudy times. The following conclu.ion. .earn Ju.tified: 

1. For medical school classes taken a. a whole, the "content" programed 
text was comparable in efficiency to conventional method, of instruction. 

For large segment, within each class the programmed text wa. somewhat 

efficient then conventional instruction. 

2. Ihe u.e of the "content" programmed text permitted a saving of faculty 
time equivalent to preparing and presenting the entire course of conven- 
tional instruction. This saving was achieved with no apparent adverse 
effect on the students 

3. The efficiency of the "composite" text in teaching "content" was coaqtar- 
able to that of the "content" text alone. 
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Study time for the "composite" text averaged 24 hours (15 for "content" 
and 9 for "application"); no comparable information was available from 
control groups of their time spent studying "application" £er se, but 
their study time on the "content" text averaged just over 18 hours. 

At present, therefore, the relative efficiency of the "composite" text 
in meeting its teaching objectives is unknown. 
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CHAPTER SEVEN 

PRINCIPAL CONTRIBUTIONS OF PROJECT 



» 

I. Development of Programming Methods . 

j An efflclent method has been developed and demonstrated for preparing programmed 
Instruction materials using a team of medical .chool faculty members and 
students as writers, editors, and critics. One per.on, apeclflcally 
trained In the technology of programmed Instruction, worked with the team, but 
did none of the actual writing and progressively reduced her Importance to 
the team as a catalyst. These techniques have already been shown to be 
readily adaptable to other situations requiring the preparatic 
materials at the graduate level. 

II. Effect of the "Content 11 T>y£. 

Hie linear text has proven to be an efficient teaching method highly acceptable 
to nearly all students who used It. The majority considered It a superior method 
of learning. Its effectiveness as a teaching device probably resulted from 
requiring the students to develop, as rapidly as possible, an active working 
vocabulary of gynecologic oncology. The student A' early mastery of the 
vocabulary apparently facilitated their learning not only from the programed 
text, but also from other sources, such as conventional reading, conferences, 
and clinical conversations with colleagues, physicians, and others. 

III. Effect of "Composite 11 Text . 

This text has been shown to be effective In teaching the "application" of 
"content" knowledge to specific Individual problems of patient care. This 
effectiveness In teaching "application" can probably be attributed to requiring 
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the students to make responsible decisions in patient management. Deficiencies 
of knowledge and errors of judgment and skill are shown to the students by 
their effects on the patient's well-being. This text, with its sequenced ex- 
perience in clinical decision making, probably directs the students toward a more 
clinical orientation, and facilitates their learning from real patients in the 
wards and clinics. 

XV. Effect on Curriculum 

By requiring the faculty to define teaching objectives concretely, this research 
exposed an unreconciled conflict in the curriculum. The minimum requirements 
of the faculty for the course, when expressed i n behavioral terms, demanded vastly 
more learning time from most students than could be made available without 
sacrificing time in other parts of the curriculum. The learning time available 
to the students, rather than the teaching time offered by the faculty, was found 
to be of paramount importance in establishing realistic objectives for a course 
within the medical curriculum. 

V* Shortcomings of Evaluation Methods . 

The project had the benefit of the best available written and oral examination! 
to measure the effectiveness of its teaching program. Examinations of similar 
excellence ari widely accepted aa measures of the professional competence of 
candidates for licensure and certification. It was found that the tests used 
in the project were often inappropriate as measures of the expressed teaching 
objectives of the faculty and were usually inadequate as measures of essential 
skills in patient management specifically included in the instructional program. 

The lack of valid, reliable criterion measurea proved to be a major handicap 
in the preparation, presentation, and evaluation of the teaching programs. 

VI. Development of New Tests . 

V/' 

Significant progress ha* bean made in the development of written testa designed 
to measure skills which cider, more orthodox examining methods had often left 
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unmeasurcd. These test, use flexible format, and have been de.lgned to pre.ent 
and measure a variety of .ophlstlcated clinical .kill.. Nine .uch teat., 23 
or more page. In length, have been developed. Several are currently In u.e 
In evaluating medical students. 

VII. Text. Produced . 

Three programmed text, have been developed. 

l * Essential, of Gyne cologic Oncology - 442 pages, 830 frames. 

2 - Applications of Gynecol ogic Oncology - 357 pages, 713 frame. - 35 cases. 

3 * Slo&ESEB£* Instru ction Method, for Ob.tetrle. .nH Gynecology. A 
Text foi Teachen - 108 pjgei. " 1 — 

Earlier chapter. In this report have presented detailed Information confirming 

the effectiveness and acceptability of the flr.t of these texts, somewhat 

less Information on the second, and none on the thitd. All three texts have 

been well received by the students and teachers who have used them. Adequate 

field testing of the latter two texts, however, must await the further 

development and validation of appropriate criterion measures. 
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(in tnis space on each of the following pages you will find the 
information in CAPITALS which is withheld from the text below) 



LOWER GENITAL TRACT 

Neoplasms of the lower genital tract - that is, the vulva and the 

vagina, fall into two groups: Cystic tumors and tumors. 

The neoplasms can also be divided into benign owl 

groups. We shall consider the lesions first, and 

take up the malignant ones later. 

(SHAME 1) 



GO ON TO NEXT PAGE 



NO 



(Biopsy and histologic study is indicated.) 



Exfoliative cytology is for screening, not for diagnosis. Diagnosis re- 
quires histopathological examination of excised tissue. Where cancer is 
suspected by gross examination, a cytology report, if positive, adds 
nothing to the indications for biopsy already present; and if negative 
should be 



1 



(SHAME MS) 










Ovarian Tumors 



^Cysti^ 

Non-neoplastic NEOPLASTIC 

* * 

BENIGN Malignant 



SOLID 

(all neoplastic) 

it ^ 

Buiign MALIGNANT 



All solid ovarian tumors, regardless of the age-group of the patient, 

are (neoplastic/non-neoplastic). Regardless of whether such 

tumors prove to be — . or malignant, they should be 

treated by 



(FRAME SIS) 
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Cystic tumors and SOLID tumors 
benign and MALIGNANT 
the BENIGN lesions 
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Not all the (cystic/solid) (underline one) tumors of the. 
ant * — are true neoplasms. 



Non-. 



.cysts worth mentioning are: 



1. Inclusion cysts and 

2. Cysts of embryonic remnants 



(FRAMB 2) 



GO ON TO NEXT PAGE 

IGNORED or REJECTED 



Since a Class V Papanicolaou smear is considered conclusive for malig- 
nancy, does a Class V smear establish a diagnosis of cancer? (Yes/No) 

(FRAME 200) 



3 
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Ovarian tumors: Cystic, solid, benign, malignan t 
Cysts: Neoplastic and non-NEOPLASTIC 



Ovarian Tumors 



Cystic 

/ \ 

Non-neoplastic 



Malignant 




(all neoplastic) 



Benign 



(FRAME 017) 



OO ON TO NEXT PAGE 




DIAGNOSIS and TREATMENT 

If YES, make your own outline. 

If NO, make your outline while you review. 



This is the end of the program. Please be sure your time-sheets are 
complete; then, if you have time, reread the “Objectives of Course”. 
It may help you to determine which aspects of gynecologic oncology 
you need to review. 



4 



(FRAME 838) 





MopUims? CYSTIC hm,ors of ,he VUL VA and VAGINA are true 



151 



NON-NEOPLASTIC cysts are: 

1. Inclusion cysts 

2. Cysts of embryonic remnants 



cysts of the lower genital tract occur as a result 

of the burying or inclusion of a portion of epithelium in the repair of 

an episiotomy or (obstetric). They may occur in 

either the or the 



(FRAME 3) 



GO ON TO NEXT PAGE - ^ 



NO 

(Cytology is for screening, not for diagnosis.) 



A Class V vaginal smear indicates that the patient has definite malig- 
nant cells in her vaginal secretions. It does not tell you that the cells 
necessarily came from a fully developed cancer, nor where the cancer 
is, nor its extent, nor its invasiveness. A diagnosis of cancer requires 
histologic confirmation. 

Question: Does a Class I Pap smear rule out cancer of the cervix? 



5 



(FRAME *10) 
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CLASSIFICATION 

AGREE 



Ovarian tumors may be solid, cystic, benign, or malignant in any com- 
bination. Cystic ovarian tumors may be neoplastic or non- 

(FRAME 818) 



GO ON TO NEXT PAGE 




If YES, do so, and keep your outline for reference 

If NO, please make your outline while you review this 
section on adnexal tumors. 

(You can find a complete outline printed in Objectives of Course) 



All ovarian tumors, regardless of size or the patients age, require 
• and 

Can you outline the management of each ovarian tumor? (Yes/No) 
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(FRAME 827) 








INCLUSION cysts occur by burying or Inclusion of epithelium in the 
repair of an episiotomy or obstetric LACERATION. They may occur 
in the VULVA or the VAGINA. 



Inclusion cysts are therefore usually found in (nulliparous/parous) 
(underline one) women. The cysts are usually small, movable, greyish 
or yellowish in color, and filled with sebaceous material or desquamated 
cells. The lining is stratified -epithelium. 

(FRAME 4) 



GO ON TO NEXT PAGE 



NO! 



Even in the hands of an expert cytopathoiogist there is a significant 



error rate for each class. 


Papanicolaou himself reported the following 


Classes I and II 


10% 


Class IV 


5% 


Class V 


2% 



Other large series have reported a considerably higher eiror rate (15%) 
for "false negatives" in Classes and 
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(FRAME Sll) 







40 PRIMARY tumors 



METASTATIC (from uterus, gastrointestinal tract, breast) 



There is no classification of ovarian tumors which satisfies everyone, 
mostly because experts disagree on the histogenesis of many tumors. 
Nevertheless, the practical problems of caring for a patient with an 
ovarian tumor require that you have a working fr, 

mind. The various classifications disagree over details, but in certain 
fundamentals of management they all 

(FRAME 618) 

GOON TO NEXT PAGE ^ 

(in effect) 

BY PERIODIC, REGULAR PELVIC EXAMINATIONS 
OF ALL WOMEN 



Can you write out, in your own words, a classification of ovarian 
enlargements (tumors)? (Yes/No) 
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(FRAME SMI 










PAROUS women are most apt to have small inclusion cysts, filled with 
sebaceous material and lined by stratified SQUAMOUS epithelium. 



Considering the etiology of these cysts, what would you suppose t he 
malignant potential to be? 

Once the diagnosis is made, what treatment is usually required? 



(FRAME S) 

PC* ON TO NEXT PAGE ^ 

ERROR RATES 
I and II 

(Note that "false negatives” are far more frequent than "false positives") 



The error rate ("false positive") for Class V smears is about 

The error rate (“false positive") for Class IV smears is about 

■n» enror rote <“■ ”) for Clan I end 



%. 






II smears is about 
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(FRAME SIS) 





SECONDARY, METASTATIC 



At least 40 specific (primary /secondary) tumors may arise from ovarian 
tissue. The ovary also tends to be the recipient of secondary, 

— tumors, especially ones arising in the uterus, 

gastrointestinal tract, and breast. 



(FRAME S14) 



GO ON TO NEXT PAGE > 



7. NOT ALL OVARIAN NEOPLASMS ARE CANCER. 

8. SOME OVARIAN CANCERS ARE CYSTIC. 

9. SOLID OVARIAN TUMORS ARISING AFTER THE MENO- 
PAUSE ARE USUALLY CANCEROUS. 



With currently available knowledge (your knowledge), how can the 
death toll from ovarian cancer best be reduced? 

(your words) 
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(FRAME US) 
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Malignant potential = NIL 
Treatment needed = NONE 

(Under some circumstances you might excise or marsupialize the cyst.) 



Gartner’s duct cysts are cystic dilatations of an emb_ remnant. 

duct is a portion of the or 

Wolffian duct. The cysts are usually located in the upper t 

and tend to be large, soft, thin-walled, lined by low cuboidal epithelium 
and filled with clear serous fluid. 

(FRAME (1) 



60 ON TO NEXT PAGE — 



Class V: 

Class IV: 

Classes I and II: 



“false positive” 2% 

“false positive” 5% 

“FALSE NEGATIVE” 10% or 15% 



A “false negative” smear in a cancer patient may indicate either: 

1. An error in interpretation by the cytopathologist 

2. An error in sampling (collecting the specimen) by the clinician 

3. Failure of the cancer to exfoliate identifiable cancer cells 

Failure to exfoliate identifiable cells is especially apt to occur in cases 
of (invasive/preinvasive) cancers, where the tumor is separated from 
the vaginal secretions by a layer of necrotic cells and debris, and thus 
the smear may show only evidence of an inflammatory process. 

(FRAME SIS) 



11 






(your answer) 

1. No treatment (for minimal, or asymptomatic disease) 

2. Endocrine therapy (for mosfr^ symptomatic cases, as a trial) 

3. Conservative surgery (for failures of endocrine therapy) 

4. Castration (surgery, x-ray) (for severely asymptomatic, 
intractable cases where all other methods have failed) 



OVARIAN TUMORS 

The ovary can be the site of an awesome array of important and 
interesting tumors. No other organ in the human body can equal it 

in the number and variety of primary and tumors 

which can be found in it. 

(FRAME 613) 



GO ON TO NEXT PAGE 



4. ALL OVARIAN NEOPLASMS SHOULD BE REMOVED. 

5. HENCE ALL SOLID OVARIAN TUMORS SHOULD BE 
REMOVED. 

6. SOME, BUT NOT ALL CYSTIC OVARIAN TUMORS SHOULD 
BE REMOVED. 



Unscramble: 

7. cancer Not all ovarian are neoplasms. 



8. ovarian cystic are cancers Some. 



9. Solid ovarian tumors arising after the menopause are usually 
cancerous. 
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Gartners duct cysts: dilatations of an EMBRYONIC remnant. 

GARTNER’S duct: a portion of the MESONEPHRIC or Wolffian 
duct. 

The cysts are soft, thin-walled, filled with clear serous fluid and are 
usually located in the upper VAGINA. 



— * J. A Cell* 



The malignant potential of the cyst is almost 

ment, for symptoms of a mechanical nature, consists of surgical 

, but since these cysts are usually asymptomatic, 
surgical therapy is — — . — ... necessary. 

(FRAME 7) 



GO ON TO NEXT PAGE 



Failure to exfoliate identifiable cancer cells may occur with 
INVASIVE cancers* (which are often secondarily irfected). 

•Especially invasive adenocarcinomas of the cervix 



Why should a false negative occur? 

Error in — 

Error in 

— of the cancer to 

identifiable cells 



by cytopathologist 
by clinician 
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CASTRATION by destroying OVARIAN function avoids injury to 
the BOWEL or BLADDER. 



i 

I 

j 

List four approaches to the management of endometriosis, and give 

j appropriate indications for each approach. 

i 

1 



3. 

4. 

(FRAME SIS) 
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1. ALL SOLID OVARIAN TUMORS ARE NEOPLASTIC. 

2. ALL NON-NEOPLASTIC OVARIAN TUMORS ARE CYSTIC. 

3. SOME CYSTIC OVARIAN TUMORS ARE NEOPLASTIC. 

(also) 

3. SOME NEOPLASTIC OVARIAN TUMORS ARE CYSTIC. 



Unscramble these: 

4. removed should All be ovarian neoplasms. 

5. removed should Hence all be tumors solid ovarian. 



6. Some, removed should be all, but not ovarian turnon cystic. 




A 



A 
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Malignant potential = NIL 

For symptoms of a mechanical nature (such as dyspareunia) treatment 
consists of surgical REMOVAL, but is SELDOM, RARELY necessary. 



How would you identify a Gartner’s duct cyst on pelvic examination? 

1. consistency? 

2. thickness of wall? 

3. where found? 

4. contents? 

(FRAME 8) 

GO ON TO NEXT PAGE ^ 

INTERPRETATION 

SAMPLING 

FAILURE to EXFOLIATE 



We omitted listing an error rate for Class III. What error rate would 
you predict? 

If the smear is reported as “doubtful” or “ ”, 

what kind of result would be erroneous? 
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Castration: 

By REMOVAL or EXCISION of 



ovaries or by X-RAY 



is most useful in patients with extensive, symptomatic 
involvement of the intestinal or urinary tracts. In such cases simple 
on of function is a safe means of therapy 



as 



compared to major resection of the 



or 



(FRAME 611 ) 




CANCER 

NON-NEOPLASTIC 



Unscramble the following sentences. 

1. ovarian neoplastic tumors are solid All. 

2. tumors All cystic are ovarian non-neoplastic. 

3. ovarian Some neoplastic cystic tumors are. 



(two ways) 
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1. SOFT 

2. THIN-WALLED 

3. IN THE UPPER VAGINA (usually) 

4. FILLED WITH CLEAR SEROUS FLUID 



— remnants is a cyst of the M»»al 



Another cyst of 

of Nude (also called a vulvar hydrocele) which is a soft cyst found in 
the inguinal region lined by serosa and formed by remnants of the 
processus - which in embryonic life accompanies the 

-ligament in the canal. This 

is a very (common/rare) cyst. 



(FRAME 9 1 
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NONE 

A SUSPICIOUS or '‘doubtful” smear is neither negative nor positive 
for malignancy, therefore, neither result would be erroneous. 

NONE 



Cytopathologists differ in the criteria they use in calling a smear “doubt- 
ful . suspicious’, or “Class III”. As a result, the incidence of malig- 
nancy in patients with Class III smears varies from laboratory to labora- 
tory. As a rule of thumb, however, you can make a working assumption 
that further diagnostic studies of a patient with a Class III Pap smear 
have about a 50-50 chance of revealing a 
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CASTRATION of the patient removes the stimulus to ENDOMETRIAL 
growth. OVARIAN function and ESTROGEN are lost. 



Castration may be accomplished by simple Q f the 

ovaries, or in patients with such advanced disease as to be inoperable 
(which is very rare), by -ray therapy. 

(FRAME 010) 



GO ON TO NEXT PAGE ^ 



Adnexal NEOPLASMS should be REMOVED. 
Postmenopausal tumors are NEOPLASTIC and should be REMOVED. 



Most solid adnexal tumors arising after the menopause are 



Most ovarian cysts under 5 cm. arising during the reproductive 
years are . 
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A cyst of the Canal of Nuck is a cyst of: 

EMBRYONIC remnants, located in the inguinal region, and formed of 
serosal remnants of the processus VAGINALIS, which in embryonic life 
accompanies the ROUND ligament in the INGUINAL canal. A RARE 
cyst. 



Thus far, we have considered the following non- . 

1. cysts 

2. Cysts of 

(a) duct cyst 

(b) Cyst of the Canal of Nuck or vulvar 



cysts: 



(FRAME 10) 



GO ON TO NEXT PAGE .-<> 



MALIGNANCY or CANCER 



From the standpoint of practical management of patients, Papanicolaou 
smears fall into two groups: 

1. “Negative” smears, which in themselves do not indicate the 
need for further diagnostic studies for cancer, and 

2. “Positive” and smears, all of which 

require further diagnostic procedures to establish or rule out 
the diagnosis of cancer. 
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For ADENOMYOSIS: 
TOTAL HYSTERECTOMY with 
CONSERVATION of the ovaries 



Radical therapy consists of of the patient. This cures 

the disease by removing the stimulus to further 

growth. The function is of course lost, as is also the 

patient s main source of the important metabolic steroid 

(FRAME 60S) 
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GRIM 



All solid adnexal tumors are and should be 



All adnexal tumors (cystic and solid) arising after the menopause 
are — and should be 
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Non-NEOPLASTIC cysts 

(1) INCLUSION cysts 

(2) Cysts of EMBRYONIC REMNANTS 

(a) GARTNER'S duct cysts 

(b) Cysts of the Canal of Nuck or vulvar HYDROCELE 



The lower genital tract is also the site of glandular cysts, some which 
are neoplastic and some of which aren't. The most common non- 
neoplastic glandular cyst of the lower genital tract is the Bartholin cyst. 

This cyst results from dilatation of the duct of the _____ 

gland as a result of distal obstruction of the ^ 

the gland. 

(FRAME 11) 



GO ON TO NEXT PAGE 



1. “Negative” Classes I and II 

2. “Positive” and “SUSPICIOUS” or “DOUBTFUL”, Classes III, IV, V 



From the standpoint of though not of progno.it, 

a Class III smear and a Class V smear are They 

both require histopathological study of excised 
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(ntAiasis) 




Conservative treatment: Total HYSTERECTOMY with 
CONSERVATION, PRESERVATION of OVARIAN tissue 



For 



(intra-uterine endometriosis) when symptoms 



are sufficient to warrant surgery, 



with 



of ovaries is usually the procedure of choice. 

(FRAME 606) 



GO ON TO NEXT PAGE 




TOTAL HYSTERECTOMY and BILATERAL SALPINGO- 

OOPHORECTOMY 



(In these rare cases who knows whether to do an OMENTECTOMY?) 



Since a diagnosis is seldom made early enough for the tumor to be still 
confined to the tube, prognosis for most patients is , 
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The most common non-neoplastic cyst of the lower genital tract: the 
Bartholin cyst, caused by dilatation of the duct of the BARTHOLIN 
(or VULVOVAGINAL) gland as a result of obstruction of the DUCT 
of the gland. 



The distal part of the Bartholin duct may become by 

scarring due to recurrent infection or blockage by inspissated secretions. 

In either event, glandular secretion continues, and a 

forms. 



(FRAME 13) 



GO ON TO NEXT PAGE — ^ 



In MANAGEMENT, Class III and Class V are IDENTICAL or 
the SAME. They both require study of TISSUE. 



All Pap smears of Classes III, IV, and V indicate the need for 
• study. 
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FIBROUS or SCAR tissue 
LESS risk 
NORMAL tissues 
SURGICAL therapy 
ENDOCRINE therapy 



In some women in whom desire for further childbearing is not a con- 
sideration, with extensive uterine but resectable adnexal involvement, 
appropriate (conservative) surgical treatment may consist of total 
with of normal tissue. 

(FRAME 607) 



GO ON TO NEXT PAGE — 



MICROSCOPIC EXAMINATION OF TISSUE 



Treatment of operable cases of tubal cancer is the same as for operable 

cases of ovarian cancer, i.e., 

and __ 



(name the surgical operations of choice) 
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The duct may become OBSTRUCTED (or BLOCKED). Secretion 
continues and a CYST fonns. 



The cyst tends to increase slowly in size, and is usually filled with a 
mucoid, clear or yellowish secretion. Malignant potential of the cyst 



is almost nil, although 

(cell type) 

gland itself are sometimes reported. 



-carcinomas arising from the 



(FRAME 13) 



GO ON TO NEXT PAGE 



TISSUE or BIOPSY or HISTOPATHOLOGICAL 



Both normal columnar epithelium of the endocervix and abnormal 
squamous epithelium in the cervix - i.e.: “new” metaplastic, or dyplastic, 
or neoplastic (cancerous) epithelia — are deficient in glycogen. They 

therefore fail to stain deeply with This is the 

basis of the Schiller test. 



O 

ERJC 

iminaffaminaa 
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PALLIATION 



ENDOCRINE therapy 
PREOPERATIVE preparation 
CONSERVATIVE surgery 



Several months of endocrine therapy make the surgery easier, by bring* 

ing about softening of adhesions and tissue surrounding 

implants, permitting the surgeon to develop planes of cleavage, and 
thus remove the diseased areas with (more/less) risk of injury to sur- 
rounding — — tissue and organs. 

Thus even when therapy is planned as the treatment 

of choice, — - — — therapy may improve the chances of a 

good result. 

(FRAME 000) 



GO ON TO NEXT PAGE ■— ^ 



(Inflammatory) HYDRO- or PYO-SALPINX or TUBO-OVARIAN 
ABSCESS 

(Hematosalpinx) ECTOPIC PREGNANCY* 

Hematosalpinx is almost pathognomonic of ectopic pregnancy. 



Ultimately the diagnosis must be made by; 

(your words) 
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Malignant potential: almost nil 
ADENOCARCINOMAS are sometimes reported. 



The chief complication of a Bartholin cyst is infection, 

of a Bartholin _ This occurs so 

Bartholin cysts warrant prophylactic treatment. 



with formation 
commonly that 



(FRAME 14) 



GO ON TO NEXT PAGE 



IODINE 



The Schiller test consists of painting the cervix with an aqueous iodine 
solution (usually Lugol’s) and observing the staining reaction. The test 
may be used to delineate the following non-staining tissues: 



1 . 



2 . 



3. 



(Normal). 

(Benign). 

(Ptemalignant). 



(Glandular epithelium) 



(Squamous epithelium) 



(Neither of these 
are premalignant) 



4. (Malignant) .. 

— (invasive or 

pre-invasive) 

(FRAME 221) 
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SUSPENSION of uterus 
LYSIS of adhesions 

EXCISION. DESTRl CTION. REMOVAL of implants 
IMPROVED fertility 
Improved FERTILITY 
ENDOCRINE therapy 



Neither endocrine nor surgical therapy of endometriosis is usually cura- 
tive. In most cases, they offer only Many clinicians 

have found therapy most useful in endometriosis as 

a pre preparation for surgery. 

•FRAME 605' 



GO ON TO NEXT PAGE 



Same as OVARIAN cancer 



Correct diagnosis is RARELY made. 



At the time of laparotomy the appearance of the unopened tube often 
suggests an inflammatory lesion, such as 



or 



or sometimes, the hematosalpinx of an 
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Current surgical treatment of a Bartholin cyst calls for marsupialization. 

A new permanent opening is made between the cyst and the overlying 

epithelium of the vaginal 

(anatomical part) 



(FRAME IS) 



GO ON TO NEXT PAGE 




1. (Normal) GLANDULAR or COLUMNAR EPITHELIUM 

2. (Benign) SQUAMOUS METAPLASIA 

3. (Premalignant) ATYPICAL SQUAMOUS METAPLASIA or 
DYSPLASIA 

4. (Malignant) CANCER (invasive or pre-invasive) 



Normal squamous epithelium of the cervix and vagina turns a deep 
mahogany color when stained with iodine. The Schiller test can be 

useo to identify epithelium which 

need not be biopsied. 
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EXCISION and/or FULGURATION of cysts and implants 
Prcsacral NEURECTOMY to DENERVATE the uterus 
Lysis of ADHESIONS 
SUSPENSION* of the uterus 
This is the name of the operation. 



Uterine 

addition to 



and 



or adhesions, in 



... — — .... of implants, results in significantly 

— _ed fertility postoperatively in patients with endo- 
metriosis. In small series of patients, this same improved 

has also been demonstrated for conservative therapy. 

(FRAME 604' 
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(in effect) 

(continuity) OVARY, UTERUS 

(contiguity) PELVIC PERITONEUM, SEROSA OF BOWEL 

AND OMENTUM 

(lymphatics) UTERUS, OPPOSITE TUBE AND OVARY, 

AORTIC, ILIAC, HYPOGASTRIC NODES 

(vascular) DISTANT ORGANS, LUNGS 



Differential diagnosis of tubal cancer, preoperatively, poses all the 

same diagnostic possibilities as does — cancer. Because 

tubal cancer is such an unlikely possibility, a correct preoperative diag- 
nosis is (rarely/frequently) made. 
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MARSUPIALIZATION OF BARTHOLIN CYST 

GC ON TO NEXT PAGE ^ 



The Schiller tost identifies NORMAL SQUAMOUS epithelium. 



The Schiller test is therefore useful in indicating the margin between 

the normal squamous epithelium and the -staining epithelium, 

which in an adequate biopsy, should be j n the 

specimen. 
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DYSMENORRHEA, division of the 
SACRO-UTERINE (or uterosacral) ligaments 
FRESACRAL neurectomy 



In addition to — and/or — — of implants, 

and presacral — to the uterus, con- 

servative surgery for endometriosis also commonly includes lysis of 

binding down the tubes, and L _ 

of the uterus in an anterior position. 



(FRAME 603) 
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DISTAL end 

PELVIC PERITONEUM, OVARY, etc. 



The spread of tubal cancer resembles that of ovarian cancer 

by continuity to 

(connected structures) 

by contiguity to 

(structures in contact) 

by lymphatics to __ 

(regional structures and nodes) 

by vacsular system to 

(name the organs) 

(your words) 
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□ 
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Marsupialization: a new permanent opening between the cyst and 
the overlying epithelium of the vaginal VESTIBULE. 



D 
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The operation permits continued drainage of the cyst contents and of 



the normal secretions of the 



.. Once 



is established, the dilated duct soon returns to normal. 



(FRAME 18) 
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NON-staining 

INCLUDED or REMOVED (in) or EXCISED (with) 



Since invasive cancer usually develops from a pre-invasive cancer, it 
should be obvious that in many cases the two lesions will 
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CONSERVATIVE surgery 
RELIEF of pain 
IMPROVEMENT of fertility 



As prolonged though often partial relief for severe dys , 

partial denervation of the uterus is often done, either by transection of 

the saero-__ - ligaments (Doyle’s operation) or else by 

pre„„_ — > — __ — neurectomy, which is the more commonly per- 

formed operation. 

(FRAME 602) 
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EndoSALPINX 



BILATERAL in 30% 



The tumor grows most commonly near the (distal/proximal) end of 
the tube, converting it to a fusiform mass, and soon spills through the 
fimbriated end onto the __ . 

I' M ' w nr i * “ 1 ■ 1 ■ ' — — ■ ■ ■ ' t 

(FRAME 813) 
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The marsupializing operation permits drainage of the secretions of the 
GLAND. Once DRAINAGE is established, the dilated duct soon re- 
turns to normal SIZE (or CALIBER). 



Because of its simplicity, relative bloodlessness, and effectiveness in 

restoring normal physiology, the operation of 

has (replaced/been replaced by) the older, much less satisfactory pro- 
cedure of surgical excision of the cyst, which was reall y 
of the dilated __ of the gland. 

(FRAME 17) 
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COEXIST or BE FOUND TOGETHER. 



Invasive cancer frequently is surrounded by a border of. 
— cancer. 
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Conservative therapy may be endocrine or SURGICAL. The CON- 
SERVATIVE approach calls for PRESERVATION, CONSERVATION, 
of reproductive FUNCTION with removal only of areas of ENDO- 
METRIOSIS, DISEASE, INVOLVEMENT. 



Excision and/or fulguration of cysts and implants is the main objective 

of surgery. Other objectives are . 

of menstrual pain and ______ 0 f fertility. 

(FRAME (SOI) 
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LEAST likely site: OVIDUCT, TUBE 
One out of 1,000 pelvic CANCERS 



The carcinoma originates as a nodule within the endo___ 

Like cancer of the ovary, cancer of the tube is lateral about 

-% of the time. 
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MARSUPIALIZATION has REPLACED the obsolete operation of 
Bartholin cystectomy, which was really EXCISION of the dilated 
DUCT of the gland. 



Ectopic endometrium may at times be found in the vulva or vagina, 
forming bluish cystic lesions which tend to enlarge and become painful 
at the time of the These cyclic changes are char- 
acteristic of Treatment, when needed, is surgical 

(name the disease) 

or (sometimes) endocrine therapy with progesta- 
tional agents. 



(FRAME 18) 
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PRE-INVASIVE or INTRA-EPITHELIAL 



A cervical punch biopsy showing intra-epithelial cancer may represent 
either: 



(1) A true _____ lesion, or 

(2) The edge of an _ — lesion 



37 



(FRAME 228) 







m 



PSEUDO-PREGNANCY 
PROGESTATIONAL 
ATROPHY, FIBROSIS 
SCAR or FIBROUS 



The conservative therapy of endometriosis may be endocrine or 

— — — . — — — surgical therapy is directed towards 

— — • of reproductive with removal 

only of the areas of 



(FRAME ( 500 ) 



GO ON TO NEXT PAGE 

INSIDIOUS onset 

The cure rate is POOR, DISHEARTENING, DISCOURAGING. 

PALLIATIVE therapy has much to offerl 

(The above applies as well to cancer of the tube as to cancer 

of the ovary.) 



THE TUBE 

The only genital organ we have not considered and the portion of tin 
female genital tract (most/least) likely to be the site of primary carci 

noma is the — — . Primary adenocarcinoma of this orgai 

occurs only about once out of every 1,000 pelvic 
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appendix b 



SAMPLE FRAMES FROM APPLICATIONS OF GYNECOLOGIC ONmT/inv 



by 



Preston Lea Wilds, M.D. and Virginia Zachert, Ph.D 



Department of Obstetrics and Gynecology 
Medical College of Georgia 
Augusta, Georgia 



Diagrams and discussion of cases (from the text 
Programmed Instruc tion Methods for Obstetrics and 
Gynecology, a T ext for Teachers ! Pages 212-220 



Copyright 1964, Medical College of Georgia 



TWO CASES 



from 



APPLICATIONS OF GYNECOLOGIC ONCOT/Viv 



D 

0 



Case #5 « UPPER CASE 
Case #14 » LOWER CASE 



0 

Q 

Q 

Q 
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UPPER CASE 



U1 



A 50-yaar old aarrlad nullipara comb to your offlca with tha complaint 
of "aor# on bar prlvataa”. 8ha a tat* a ah* flrat notlcad tha laalon 
about two waaka pravloualy. Examination ravaAta a palnlaaa, firm, 
ulearatad araa 2 ca in diaaatar Involving tha right half of tha elltorla 
and har adjaeant labluai minus. Lyaph nodaa In tha inguinal raglona at* 
modarataly onlargad and not tandar. Thar* ar* no othar poaltlva phyai- 
cal finding* « 



Co on to U 2 for llluatcatlon. 



D 

Q 

0 



UI 



D 



LOWER CASE 



U 



A 45-yaar old woaan raturna to your ear* aftar a yaar'a abaanca couplalw 
lng of raetal blaadlng of a aonth'a duration. Two yaara pravloualy, 
whll* under your car* ah* had raealvad radiation traataant for aquaaoua 
call earelnoaa of tha oarvlx Stag* ZXa, Har followup for tha flrat 
yaar following traataant had baan uneventful. Sb* daniaa any ayaptoaa 
during tha paat yaar axcapt tha davalopaant during tha paat aonth of 
brl^it rad raetal blaadlng mostly with bowal movements. How would you 
approach tha further management of thla patient? (Juat writ* down your 
nartt atop.) 



D 
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Writ* your anewar, than turn to L 2, 
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U 2 




Nuu go to U 3. 



From the list below please mark the answer which most closely corresponds to 
your own, then turn to the 1£WKR PACK indicated, 

o Review previous records and obtain additional history, I# 3 

a Perform a general physical examination, L 4 

a Perform a pelvic and rectal examination, L 5 

□ Perform a proctosigmoidoscopy, L 6 

□ Order some diagnostic studies, I* 7 
a Do non, of th, li«t,d option,. L 8 
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U 3 



Which one of 
management? 



£ e J° llowin8 wou ^ y° u consider the most appropriate initial 
(Mark your choice, then turn co the UPPER PAGE indicated)^ 



An incisional or punch biopgy of the edge of the lesion. U 6 
An incisional or punch bi psy with other tests or studies. U 9 
An excisional biopsy of the lesion. U 12 
VDRL, Kahn, or equivalent test for syphilis. U 13 
Other studies, without biopsy, U 14. 



REVIEW PREVIOUS RECORDS AND OBTAIN ADDITIONAL HISTORY 



Summary: 



Patient, a 43-year-old unmarried nullipara had been referred to you 
two years previously by another physician who in the course of a 
routine physical examination had noted* a cervical lesion and had 
obtained a punch biopsy which was reported as invasive gquanwus cell 
carcinoma. The lesion involved the entire cervix and a sma ll amount 
of both lateral vaginal fornices and was staged as Ila, After a 
complete history, general physical exam and the usual workup for a 
cancer patient, including cystoscopy proctosigmoidoscopy, IVP, barium 
enema and metastatic sur-ey, all of which was within normal limits 
a decision was made to treat the patient by radiation therapy. This 
treatment is summarized on L 10. V 
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1, 

2 . 

3. 

4. 
5* 

6. 

7. 

8, 

9. 

10 . 

11 . 



12 . 

13. 

14. 

15. 

16. 

17. 

18. 

19. 

20 . 
21 . 
22 . 

23. 

24. 

25. 

26. 

27. 

28. 

29. 

30. 

31. 

32. 

33. 

34. 

35. 

36. 

37. 

38. 

39. 

40. 



PHYSICAL FINDING? 

Unremarkable for adult woman 
Pap smear taken See Report 
Not enlarged 
Atrophic 

Stenotic, no lesions visible, 
indurated posterior fornix 
Marital, nalliparous 
Normal size, fixed in vaginal 
axis 

Atrophic, flu h with vault 
Not palpable 

3 cm. granular mass in anterior 
wall at level of cervix 
Rectovaginal septum thickened in 
upper part. Parametria soft, 
pliable 

Intact, no lesions 
Symmetrical, normal 
Grey, normal 

Pupils RRERLA, EOM & fundi 
normal 

Canals clear, membranes intact 
Unremarkable 

Edentulous, no lesions si. 

mucous pallor 

No masses or lesions 

Not enlarged, NSR, no murmurs 

Clear to P & A 

Supple, no masses 

Midline 

Not enlarged 

Undistended 

Flat, radiation ski., changes in 
suprapubic area 
Not felt 
Not felt 
Not elicited 
37.1°, 80, 20, 130/80 
140 Iba. 

Adult woman in apparent good 
health 

No enlarged nodes 
Symroetxical, no abnormalities 
noted 

Within normal limits 

Well-formed 

Phyeiologic 

Present and equal 

Radiation changes over sacrum 

Absent 



CODE 



■ Indicated , required by presenting problem. 

* for *crecniug or completeness of evaluation. 
« Probably useless but harmless in this case. 

* Contraindicated , not in the patient's interest, 
" ^auriyng . bogus answer. 



L 4 





GENERAL PHYSICAL EXAMINATION 


189 | 

L A 


(I) 

(I) 

(I) 

a> 


Please check the parts you wish to 
examine and look up the findings with tilt 
corresponding code numbers in the column on 
the left. 




(i) 

a) 


TPR, BP 


30 ( ) 


Hgt., W Rt. 


3 1 ( ) 


a) 

d> 

a) 


General Description 


32 ( ) 


Skin 


"12 ( ) 


Lymphatics 


33 ( ) ! 


Heed and face “ 


13 ( ) 


«> 


Hair 


14 ( > i 


Eyes 


"l§( ) 




Ears 


16 ( ) 


a) 

a) 

(R) 

(R) 


Nose 


1 7 ( ) 


Mouth, throat, teeth 


18( ) 


Neck 


22( ) i 


Trachea 


23 ( ) 


Ihyroid 


24 ( ) < 


(R) 

(R) 

(R) 


Vessels 


25( ) 


Chest 


34 ( ) 


Breasts & axillae 


19( ) 


Heart ~ 


20 ( ) 


(R> 

(R) 

(R) 

(R) 

(R) 

(R) 

(R) 

(R) 


Lungs 


21 ( ) 


Abdomen 


26 ( ) 


L3K ~ 


27 ( ) 


Masses 


28( ) 


Tenderness ^ 


29( 


Pelvic examination 


9 f 

«* 4 


) 

) 


Hair distribution 


1( 


Ext. genitalia ■ 


4( ) ! 


SUB Glands 


3( ) I 


(I) 

(I) 

(I) 

(I) 

(R) 

(R) 


Introitus & perineum 


6( ) 


Vagina 


5( ) 


Cervix 


8( ) 


Uterus 


7( ) i 


Adnexa 


9( ) | 


Roctal 


in > 


Sphincter 


12( ) 


(R) 


Massts 


10( ) 


Back 


39 ( ) 


(X) 


Extremities 


36 ( ) 


(R) 


Pulses 


38 ( ) 


DIRS ~ " 


"37 ) 


(R) 


Neurological 


35 ( ) 


! 


(R) 

(R) 






i 

| 


(R) 

(R) 

(S) 


DIRECTORY (your next .top) 

Old records + history. PAGE L 3 
Proctosigmoidoscopy. PAGE L 6 
Diagnostic studies. PAGE L 7 
Your diagnosis + treatment. PACE L 11 


i 

1 

i 







f 



ISO 

HNDXHG8 

1. Unremarkable for adult woaum (X) 

2. Pop mir taken. See report (X) 

3* Hot enlarged (x) 

4. Atrophlo (x) 

5* Stenotic, no lesions vis lb la, 

IWurated posterior fornix (I) 

6, Marital, nulllparous (X) 

7. normal alia, flxad In vaginal 

axla (x) 

6* Atrophlo, flush with vault (|) 

9, Not palpable (x) 

10. 3 cm. granular mass in antarlor 

wall at laval of oarvix (X) 

11. Xaotovaglnal septum thicks nad In 

«PP*r part. Parametria soft, 
pllabla. (x) 

12. Intact, no laalona (x) 



COW 



vtuna & uncial uamxmaiion 

(**•• findings In this ease whan dona 
uadar ganaral anasthasla) 



L5 



Cbaok tba parts you wish to examine and 
look up tha findings with tha corres- 
ponding coda auabtrs In tha column to tha 
laft. 



Pelvic examination. 



Hair distribution 
ixt. genitalia 
SOB glands 



Feriaeum & lntroltua 

Vagin a 

Cervi x 

Uteru e^^ 

Adnexa” 



, 2 < > 
. K ) 
,4< > 
. 3( ) 
6 < ) 
J5( ) 

«< ) 
,7( ) 
9< ) 



Rectal 



Sphincter. 

Hesses 



_H( ) 
.12 ( ) 
.10( ) 



DMCXWf (jrour next step) 

Review of raoorla l history. PACK L 3 
General physios 1, PACK L 4 
Proatoslgaoldoroopy. PACK L 6 
Diagnostic a tulles. PACK L 7 
Your diagnosis and treatment. FAQK L 11 



2 ■ jPd fraud, required by presenting problem. 

* " MMIMi for screening or ooap lateness of evaluation* 
D - Probably JUftUu but harmless In this ossa. 

C - fiMttfttodtolttl, not in patient's Interest. 

* " Sourlous. bogus answer. 



15 



0 

Q 

Q 

D 

Q 

D 

D 

D 

0 

D 

a 

a 

a 



o 



o 

ERIC 
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U 6 



INCISIONAL OR PUNCH BIOPSY 



Pathologist's report: Received in formalin is a small fragment of ulcerated 

tissue. The microscopic picture is typi-al of a granulomatous ulcer of the 
epithelium, with infiltration by lymphocytes and plasma cells and Increased 
cApllli^ proliferation and fibroblastic activity in the ulcerated area. 
Surrounding this, there is subepithelial infiltration with neutrophilic 
leukocytes and the epithelium shows elongation of the rete pegs. 

Diagnosis: Granulomatous ulcer of the labium minus ;nd clitoris. 

List the diseases which might produce the above histologic picture. 

1 . 



2 . 



3. 

4. 



Write your answer, then turn to PAGE U 8, 



U 6 



PROCTOSIGMOIDOSCOPY 



Findings: Sphlacler: Intact 

Lumsn: clear to a depth of about 10 cms. at which point it is 

partially obstructed by a mass of friable red granular tissua which 
bleeds on contact. Punch biopsies were taken from this area (Sea 
diagnostic studies PACE L 7.) 

NOTE: It was impossible to advance the sigmoidoscope pass this area of 
partial obstruction, therefore the examination was discontinued. 



DU EC TORY 




(Your Hext Step) 




Old record* and history* 


1 3 


Physical examination* 


It 4 


Pelvic & rectal examination* 


L 5 


Diagnostic studies* 


L 7 


Your diagnosis and treatment. 


L 11 





r 



D 
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1. 

2 . 

3. 

4. 

5. 

6 . 

7. 

8 . 

9. 

10 . 

11 . 

12 . 



13. 

14. 

15. 



16. 

17. 

18. 

19. 

20 . 
21 . 

22 . 

23. 

24. 

25. 



RESULTS 



Poeltlve (blood) negative (OCP) (1) 



Negative film 
Not dona 

No bony abnoraalltlea 
Radiation changea only 
Apparantly normal function and 
anatomy 

No abnormalltlaa noted 
Not dona 
Not dona 

Within normal llnlta 
13 mgpff, 

Hgb. 10, Net. 31, WBC 8,000, 
differential normal, amaar - 
Iron daf. anemia. 

No abnormalltlaa noted 
See PACE L 5 for flndinga. ' .. 
Partial obatructlon from Intrinsic 
maaa 10 cm from anuef otherwise, 
normal (jj 

Sea PACE L 6 for flndinga 
alao blopay report thla page (1) 
0, poaltlve (m 

Negative flj) 

Squamoua cell carcinoma (I) 

115 mgnfc (r) 

Sp. G. 1.022, pH 5, S6A nag., 
micro, negative (r) 

(mEq) Na 136, K 4.4, Cl 102 (U) 

Mild oateoarthrltlc change a, no 
oaaeoua lealona (u) 

Non reactive (r) 

Claaa X, negative (X) 



( 1 ) 

(C) 

(U) 

(U) 

( 1 ) 

(U> 

00 

(C) 

(R) 

(R) 



(R> 

(U) 

«) 



CODE 

X - lad ice ted , required by preaentlng 
problem. 

R n Routine, for acreenlng or complete' 
naaa of evaluation 

U « Probably uaeleae but harmleaa In 
thla caae. 

c " (tontrtlndlcatafl. not In patlent'a 
lntereat. 

t ■ Spurious. bogua answer. 



DIAGNOSTIC STUDIES 



L 1 



Plaoae check the Itema below about which 
you would like Information, then look up 
the reaulta with the correaponding code 
numbera in the column on the left. 

Chemlatrlea (Blood, serum) 

Bilirubin, direct, total 



Glucoee, 2 hr. poatprandlaT 
Electrolytea, Na, K, Cl 
Urea Nitrogen (BUN ) 



8 ( ) 
J0( ) 
_ 22 ( ) 
_H< ) 



Clinical & cyto-pathology, aerology 
Stool for blood, OC P 1( ) 

Vaginal Pap amea r 2 3( ) 

VPR L 2 4f ) 

Hematology 

Blood Group & R h I 7f ) 

C8C 12 ( ) 



Urine teata 
Urlnalyala, complete. 

X-rays 

Abdomen, uppe r 
Barium enem a 
Cheat — — — 



_ 21 ( ) 



Cholacyatogram_ 

GI aerlea 



Pelvla, AP and lateral 
Pyelogram (IVP) 

Skull 



Spine, thoracic, lumber 



_18( ) 
JL5( ) 
_ 2 ( ) 
_ 8 ( ) 
_ 7( ) 
_4( ) 
_ 6 ( > 
_13( ) 
>23 ( ) 



Procedurea and Surg. Pathology 
Blopay bladde r^ 

Blopay carvix_ 

Blopay rectum 



Blopay vagina, vulva 
Cyatoacopy. 



Glucoee tolerance teat 
Electrocardiogram 



Examination under aneatheala_ 
Proctoalgmoldoacop y 



_ 9< ) 
_ 5( ) 
_19( ) 
_3( ) 
_ 6 ( ) 
_ 8 ( ) 
_ 10 ( ) 
_14( ) 
16 ( ) 



DIRECTORY - (your “next a tap) 

Old recorda & hlatory, PACE L 3 
Phyalcal examination. PAGE L 4 
Pelvic & rectal examination. PAGE L 5 
Proctoalgmoldoacopy. PAGE L 6 
Your dlagnoala and treatment. PAGE l 11 



L 7 



D 

D 

D 

D 

a 

0 

D 

0 

D 

a 

o 

a 

D 

D 



o 

ERIC 






X. 



* X I 








1. SYPHILIS 

2. CHANCROID 

3. GRANULOMA INGUINALE 

4. LYMPHOPATHIA VENEREUM 



JjE* S? w*,.? 0 * 1 * which you may have mentioned, perhaps quite 

o? r Jt!-!i y e b t *?* * bov * four ,r * th * • 3 *«ntiel one*. if you mieeed any 9 
of theae four, pleaee turn directly to U 20 and complete ite aureaetion* After 
you have don. ao (or before, if you li.ted the fcK.“fSw 
proceed directly to PAGE U 22, rrecciy; 



U 8 



DO NONE OF THE LISTED OPTIONS 



L 8 



What would you do ina>:eed? 



Write your enawer, then turn to L 9, 



♦ 






ERIC 






INCISIONAL OR PUNCH BIOPSY WITH OTHER TESTS 






Very good, other te>t> for whet? Llet Ihe dleeeeee you would teat for by (1 

naan a oth»r then « or In addition to - blopay. U 

:• === B 

u. 

Write your an » war, then go to V 10. Q 



V 9 



(Your anawer) 



Thanh you for your reaponae. It aay be helpful in further revlalona of thla 
programed text. Por the preaent, however, pleaae return to L 2, and aelect 
one of the optlone Hated. 






L 9 




i 



Q 

9 

9 





o 

ERIC 
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0 

0 



U 10 

4 

1. SYPHILIS 

2. CHANCROID 

3. GRANULOMA INGUINALE 

4. LYMPHOPATHIA VENEREUM 



There are ocher possible diseases which you might want zc consider - Examples: 
Lipschutz* ulcer, vulvar impetigo, tuberculous vulvitis, mycotic ulcer, etc. - 
but the above four venereal diseases are essential ones you should have listed. 

If you omitted any of these four, pleaae turn directly to U 20 and complete its 

assignment. After you have done so - or even before, if your list was correct- 

list the tests you would do in addition to biopsy: 

1 . ___ 



0 



2 . 

3. 

4. 



Write your answer. 



then go to U 11* 



U 10 



y 



D 

0 



L 10 

Summary of radiation therapy: 

Patient received two 70-hour radium applications two weeks apart. Each 
application consisted of a medium tandem loaded with a 15 and a 10 mg. radium 
source and two medium sized vaginal ovoids (Manchester type) each loaded with 
a 20 mg. radium source. Following the completion of the radium therapy, the 
patient was given external deep therapy to the pelvic walls (parametrial 
technique) and received a total of 3400 r tissue dose over a four-week period 
using a supervoltage (2mEv.) x-ray machine. 

Total tumor dosages (x-ray and radium combined): 

Point A: 8,000 r 

Point B: 6,000 r 

If you find that the above summary of the radiation treatment ia unclear to you, 
please review: 

FRAMES 321-361 in the programmed text Essentials of Gynecologic Oncology. 
Afterwards you should continue with this case presentation. 



DIRECTORY (your next step) 


General physical examination 


L 4 


Pelvic & rectal examination 


L 5 


Proctosigmoidoscopy 


L 6 


Diagnostic studies 


L 7 


Your diagnosis 6c treatment 


L 11 



L 10 



196 



U 11 



i 



Syphilid • DARKFIELD examination of lymph from ulcer 
Chancroid * SHEARS from ulcer 

Granuloma inguinale * SMEARS from ulcer or (from biopay specimen) 
Lymphopathia venereum * FREI teat 



t 



* 



Q 



If you made errors in this list, please turn to U 20 and complete the 
suggested assignments. After you have done so (or before, if you didn’t 
need the review) turn to U 21 for your pathologist f s report. 



» 



Q 



i 



YOUR DIAGNOSIS AND TREATMENT 



L 11 



0 



In your work-up of this patient you should have established whether or not 
this patient has recurrent carcinoma of the cervix and if so its location 
and extent* What is your diagnosis? 



Q 



i 



On the basis of the information 



you have, what is the Stage ? 



Urite yovr answer, then turn to L 12. 



0 



* 



0 



o 

ERIC 

fa 



to 












EXC I SIGNAL BIOPS; 



1*7 

U U 



» 



l 



l 



D 

0 

D 

0 

D 

Q 

Q 



Pathologist's report: Received in formalin 13 a 2 X 2 X 5 cm. pie;e of tiieuo 

labelled "clitoris". There is > 1 cm. shallow, firm ulcer involving most of the 
right naif of the specimen. 

Microscopically, the picture is typical of a granulomatous ulcer of the 
epithelium, with infiltration by lymphocytes and plasma cells and increased 
capillary proliferation and fibroblastic activity in the ulcerated area. 
Surrounding this, there is an infiltration with neutrophilic leukocytes, and the 
epithelium shows elongation of the rete pegs. 

Diagnosis: Granulomatou3 ulcer of epithelium (clitoris) 

Your next step would be: 

P Get a 7DRL or Kahn test. U 13 

□ Request the pathologist to use special stairs for spirochetes and Donovan 
bodies, U 18 

□ Wish you had studied the lesion more thoroughly before you removed it. IJ 2 

Q Do none of the, things listed, U 16 

Mark your choice, then turn to the UPPER PAGE indicated, 

U 12 



(Your Diagnosis) L 12 

Stage IX, with recurrence » post* radiation 

The original staging of a patient with cancer remains unchanged regard less of 
the later clinical course, (Obviously, moat patients who are destined to die of 
cancer of the cervix eventually develop findings which if they were present at 
thte iirae of the initial evaluation, would hsva put them in Stage IV, but they 
still retain their original staging)* 



What therapy would you offer this patient? 
a Surgery. L 13 
a Radiation*. L 14 

□ Chemotherapy* L 15 

□ None of the above lioted therapies, L 16 

Mark your treatment of choice, then turn to the LOWER PAGE indicated. 



i 



i 








1 
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U 13 



VDUi OR KAHN TEST 



laboratory reporta: Non-reactlve 

A negative aerology la an expected finding In the preaence of a prinery chancre. 
Your etudy neither rulea ayphlllr In or out. cnancre. 



Pleaae turn to the PAGE U from which you cane. (0 3 or U 12) 



U 13 



SURGERY 



L 13 



I 

i 

I 

B 

Q 



B 



Pleaae aelect the aurgical procedure which 
then turn to the PAGE Indicated. 



aeama the noat appropriate to you, 



p- 1 Exploratory laparotony. L 18. 

a Radical hyoterectony and pelvic lymph node dlaaectlon. L 19. 
[~*~1 Posterior pelvic exenteration. L 20 
r~| Total pelvic exenteration. L 21 
□ Colostomy, L 22 

[~1 Non« of the op#r«tlon« L 23. 



L 13 



Q 

Q 

D 



o 

ERIC 



A 









I 
fi 

I 

i 

! n 

II Pleaee li»t which atudiee. 



t 



a 

Write your answer then go to U 15. 



t 



Q 



I 



Q 



♦ 



i 



Q 



I 



I 



U 14 




RADIATION 



L 14 



Your cholca of radiation to traat thla patlant euggeete that you perhap. filled 
to inveetlgate har past treataent coopletely or that you naad to know aorc about 
radiation tharapy In tha tr.ata.nt of advanced cancer of th. cervix. A brief 
review of title eubject can be found In: 

FRAMES 321-361 of Keaentlale of Cynecoloalc Oncolotv 

FRAMES 317-320 In the aaaa text cover the applicability of radiation 
therapy to thle patient. 



After you have reviewed the above eectlone, plea*, return to L 12 and chooee 
a nor. appropriate reaponee. 



L 14 



O 

ERIC 



» - - 









?oo 



(Your Answer) 



U 15 



diagnosis of ulcerative lesiona of the vulva almost olwaye requires tho uce 
** *otr<? laboratory aids, fuctuding biopsy. 

If you would liVc to review this subject, turn to PACK U ?0 and follow 
lt n directions. 

J f not, plersr chr^se nrothe- reproach on PACE t? l. 



I 

B 

I 

i 



0 



'JtF.Vrtl'Tm 



•' 15 



I is 



f hcwjthernpy tor treataeni ot •rnee? cf the cervix la nfcill In an expeiirentnl 
str*e Cor palliative nurposan wily. Tbla patient haa a leilen which rnv ? ti>» 
*>e ci'rS’e, Pleaae return *• * ’ ir .ini* c»i«oi' rnoth<r tetonr- 



D 

D 

D 

D 



! 15 



D 

a 






w 



W 






o 

ERIC 



| tb 



i 

i 

i 

i 



j 



n 



a 



o 



D 

D 



DO NONE Of THE THINGS LISTED 



M**m writ# what you would do. 



NONE Or TO* TOE RAFIKS LISTED 



Wmh iuu#at your plan of — nafint. 



Writ# your anavtr, then turn to L 17. 



U 16 



201 



U 16 



L 16 



L 16 







* 







0 

a 

o 

D 



a 

a 

D 

0 

0 
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U 18 



SPECIAL ((liver) STAINS for SFIAOCHITII and DONOVAN 10DIU 



toper t: Negative 

Hhat dlaaaaa ara Donovan bod la » associated with? 



Can tha pathologlat dlagnosa chmcrold by studying tha histologic aactlon ha has? 



What la tha cauaa of chancroid? 



v#n, 5!?! th * P«tlant hava (excluding all thoaa already 

■an t Ion ad - syphilis, chancroid and granulona Inguinale)? * 

(naaw tha dlaaaaa) 

What la tha cauaatlva agant of thla dlaaaaa? 



Wrlta your anaveri, than go to U 19. 



U 18 



L 18 



EXPLORATORY LAPAROTOMY 



Find Inga: 



Tha bowel In tha palvla la ahrunkan and partially davaacularlsad 
aa a result of tha radiation tharapy. Thara ara no palpably en- 
larged aortic or palvlc lyaph nodaa. Tha adnaxa araanall and 
atrophic. Tha poatarlor surface of tha utarus and cervix la fused 
to the anterior rectal wall In a solid tuiaor Mass, which however la 
traaiy Mobile. Thara la no palpable paraaetrlal or bladder Involve* 
Mant. 



Please return to L 13 and choose an additional surgical procedure. 



D 



L 18 






GRANULOMA INGUINALE 
PROBABLY NOT 

DUCREY BACILLUS (Hemophilua ducreyi) 
LYMPHOPATHIA VENEREUM 
A FILTERABLE VIRUS 



If your aniwera to all of the above are correct you probably can name the 
remaining diagnoatic teat you ahould perform on your patient. 



(Name the teat and turn to u 24.) 



If you mitaed any of theae anawera (or don’t know the name of the teat), pleaae 
turn directly to U 20 and carry out ita instruction*. 



i 

I 



| 



i 



U 19 



L 19 



RADICAL HYSTERECTOMY AND PELVIC LYMPH NODE DISSECTION 



Thla ia an appropriate operation for many caaea of carcinoma of the cervix. 

It la not appropriate in thia caae for reaaona which ahould have been determined 
preoperatlvely. Pleaae return to L 2, and reevaluate your patient more thoroughly. 
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D 
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U 20 



5l$ 2£ Vu ansss sr— *• *• — 



****** 34*45 in IlHfltUli of Crnacoloelc OpcoIoty 

thi f After you bam dooa so, return to page 

which dlractad you hara (battar writ# tba amber In now). ~ ' 



(j wi 



L 20 



POSTERIOR mVZC RXEHTRRATION 



Ihia would be appropriate aaoagaaant in this c an. Tha final daclalon to 
parfora this operation, however, could only ba Mda at tha tlna tha abdoaen 
waa opanad for exploration. A report of tba operative findlnga la available 
on l 18. You ahould refer to It If you have not already dona ao. You My 
thio go on to FACE v9 # 



0 



L 20 
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U 21 



Laboratory report* : 

1. Darkfield examination for apirochetea: Negative 

2. Smear* for hemophilua ducreyi and for Donovan bodiea: Negative 

3. Biopay of margin of ulcer: 

Mlcroacopic: Thera ia granulomatoua ulcer of the vulvar epithelium, with in- 
filtration by lymphocyte* and plaama cell*, and increaaed cap- 
illary proliferation and fibroblaatic activity in the ulcerated 
area. Surrounding thia, there ia infiltration with neutrophilic 
leukocytea, and the overlying epithelium ahowa elongation of the 
ret* peg*. 

Diagnoaia: Granulomatoua ulcer of the labium mlnua and clitoria, etiology to 

be determined. 

The one remaining indicated teat in your patient waa poaitive. Hhat waa the 
teat? 



The teat and it* raault are on U 25. 



U 21 



TOTAL mVIC KXKMIUATION 



L 21 



Tha deciaion to perform a total pelvic exenteration (that ia, removal of all 
pelvic viacera including rectum, vulva, vagina, uterua, tube*, ovariea, bladder 
and urethra) can uaually be mad* only rx the time the abdomen ia opened for 
exploration. Sometime* the finding* will auggeat that a lea a formidable and 
mutilating procedure may auffiea. Total pelvic exenteration carriea with it 
an operative mortality of about 25%. If a leaaer procedure will aerv* the aa*M 
purpoae of curing the patient, it ahould be conaidared. 



Heaae turn to L 18 and review the operative finding*. 



L 21 









m 



0 

0 

D 
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U 22 



In your patient, who has a granulomatous ulcar of tha vulva nraauaaM* nr 
venereal origin, what test would you use for; prwumsbiy of 

Syphilis? 



Chancroid? 



Granuloma inguinale? 



Lyaphopathla venereum? 



Writs your answers, then go to U 23. 



U 22 



L 22 



COLOSTOMY 



procedur * to divert tha facal stream If the 
p . lent s pelvic tumor Is unreaeetable . What type of colostony would you do? 

□ Sigmoid Colostony. L 25 

□ Left transverse colostony. L 26 

Mark your answer, then turn to the LOWER PAGE Indicated. 



L 22 



O 

ERIC 
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U 23 

Syphilis: DARKFIELD examination £ spirochetes 

Chancroid: SMEARS and/or culture of the leaion for Hemophilus Ducreyi 

Granuloma inguinale: SMEARS from the lesion for Donovan bodies 

Lymphopathia venereum: ? ? 



The best smear for the Donovan Bodies of granuloma inguinale is made from a 
fresh biopsy speclmen f but in this case, your biopsy was submitted to the 
pathologist in formalin, and hence this opportunity was lost. The smears 
obtained from the lesion were negative for the Ducrey Bacillua and for Donovan 
bodies. The darkfleld exam was negative. 

What other test should have been done? 



(If you don’t know, see U 20 before proceeding to U 25.) 



U 23 



L 23 



NONE OP THE OPTIONS LISTED 



What surgery would you do? 



Wtlte youi answer, then turn to L 24. 



L 23 
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U 26 



\ 



GO ON TO THE LOWER CASK ON THE BOTTOM OF PAGE L 1 



$ 



1 



j 



0 

Q 



♦ 



0 



♦ 



♦ 



U 26 



L 26 



LEFT TRANSVERSE COLOSTOMY 



Ih« above operation would permit a coloatomy to be done wall away from 
th# area of radiation Injury and devaacularlsatlon. It would be the 
procedure of choice in a patient with an unreacctablc pelvic tumor in- 
volving the rectum and producing aevere bleeding. It would be tragic, 
“"■""fa to cond#B “ thia Patient to death with a palliative operation 
without being certain that she la indeed Incurable. At the very lcaat 
an exploratory operation ahould be done to eatabllah thla fact. 



Pleaae turn to L 18. 
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MOSLEMS ZK WRITING CASK PRESENTATION PROGRAMS 
QUESTION: How do yoo keep track of tba pages? 

ANSWER: Hake a flow chart aa you wrlta. 

Exaaple: 

*««»•• your flrat paga presents tba case, but calla for 
no wrlttao response, wake a clrcla and nuabar It. 

© 

fuppoaa tba aacood paga also calla for no wrlttao raapooaa. 
Do tba aa* a thing, and add aa arrow. 



0 - - > 0 

Suppose tba third paga baa a Multiple choice question which 
branches to several enawera on different pagaa. Add the 
new paga and aoka arrows to all tba br cache a. 



S'* 

0T>© — ♦ 0 "> > 




Suppose that sous of tba branches call for written responses. 
Use squares to Identify then. 
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mm korlBMtal°l^l **** ** * * mp«MH *11 on tbt 

fMW horlfootal 11—. wick tonm* progress from left so right. 




to •• i cwyt i M t solution to tbo problsw, 
tat *oth K«**4 borlaon tally frow hit to right F ^ 
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tottjjffftk cm*, tbsrs wn tfars* tracks which Mtoi solution of th« 




Dlsgrsw of "right «niwr N branch**, Vppsr Cam. 



but hii^dlMdvMtag**^ °°* ** pr * hm4, *•* otb « two w*r* scctptsbl*. 
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Hb«n tbaraara branch#* rapraiaotlng *rrooaou* nncoHi, l«t than b* 
£SSIISir , *teO lr * 0tl00 * £thl1 eb “ horlBOOt * 1 » utt to right (up, down, 

©-^KD“^0^S ^ — > *tc. (right track) 
Tvy@~~> fi-Q > ate. (right track) 

I V@Nil 
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In thi Up?«c CIM, it la uaad ool, ooct, but tbara ara nany tafarrala to it. 
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Hu nM arrow Co Indicate aaoh dlraccioa cba iMwc mt taka la workiac 
tbroaik Cba problaw. tom arrow win ba doubla-aadaT Mara la .arc 
of cba Dppar Caaat 




IMa looka ooaplleacad bat bacowa aaaalngfal wfcaa you cry co writa a caaa 
M0X <l 2l7f tr * C * * cowrM ^roagh a yroblaw. Plaaaa go oa Co 
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DIAGRAM OF UPKR CASE 
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m sens e i tm , nf lftgtr Cm , On eh* opposite peg*, th*r* 1* a diagraa 
laballad "Upper Caaa." It llluatrataa eh* different path* which efaa atudant 
way taka In working through thla problea. in th* diagraa, the atudant' a 
progress toward* solving eh* problaa la represented by horlsontal aovaaent 
froa left to right baginning on Fraa* 1 and ending on Fraaa 24 or 25. 

Fraae 1 defines th* problaa. Fraaa 2 llluatrataa It. On Fraa* 2 th* 
student la asked to choose his diagnostic studlss. He aay ordsr thea oo* at 
* tlaa or he aay order a whole battery of thaa at one*. Ha is given five 
courses of action to choose froa. Three of these lead to horlsontal tracks 
which aay lasd to an eventual solution of th* problaa. If th* student 
reaches this solution, h* learns vhathar th* cours* b* choe* was an efficient 
on* or whether th* sane result alght have bean achieved by another cours*, 
with lass wasted effort or perhaps with less autllatlon of the patient. 

The student who does not follow one of these tracks but takes a different 
branch aay be slaply reprlaanded or aay be given raaedlal instruction. This 
reaedlel Instruction aay b* contained In this text Itself or it aay consist 
of * referral to Instruction elsewhere. In this case, the student who 
aakee certain errors is referred to Fraaa 20, which in turn suggests that he 
read th* eleven fraaes on diagnosis of vulvar lesions in our linear text. 
Afterwards, tho student is requested to tek* up th* problea at th* point 
where he left the track. 

W* selected this problea for presentation because It ***** to us to be a 
fairly clear exaaple of how the practical consequences of selection and 
sequence In ordering certain diagnostic studies can b* taught to th* student. 
In our opinion, thle type of presentation teaches th* student soatthlng aora 
than le taught In our linear text which, by th* way, teach** the atudant to 
answer axealnatlon question* on this subject quite well, W* anticipate that 
teaching by neene of prograaaed cae* presentations is aora apt to alter the 
student's perforaence In practice thaa It Is to alter his psrforarac* oo the 
. “■'■••I typ* of e real nation. 






mm 9i urn sm 



ti 




tlflfifK 

o * Mo writtM rttpooM 
CH 3 * Wrlttoo rupoui 

□a * D«t«-**tbottiif fetm 

CD ■ Mofotrin* from 

921 - 3(1 - ta«M i a othor fxt 



t 



0 

D 

D 

a 

a 



n 



y 

H 



u 



fi 

y 











Discussion of Lower Case . The lower caae also has a diagram (see opposite 
page). From the number of arrows and lines on it, it is obvious that the 
lower caae is a complicated one. The problem as stated in Frame 1 is that 
of a patient with a complaint of rectal bleeding with onset several years 
after radiation therapy for cancer of the cervix. The case is designed to 
teach the student the consequences of his diagnostic workup (whether it is 
adequate or inadequate) on his. choice of therapeutic procedures. Frame 1 
presents the problem and asks the student how he would handle it. 

Frame 2 gives the student a half dozen options for collecting 
additional information about the patient. The next eight frames give the 
student an oppe*. amity to learn a great deal about the patient. The format 
forces him to collect the information piecemeal, item by item, but it gives 
him freedom to acquire this information in any order he chooses and to uoe 
his own judgement as to when he has enough information to mansge the patient. 

On Frame 11 he is asked to write down his diagnosis. He is not told 
immediately that his diagnosis is correct or incorrect. He proceeds on 
Frame ,12 to consider various therapeutic possibilities. These include 
surgery, radiation, chemotherapy, and other forms of treatment. For 
inappropriate therapies, the student is given remedial instruction either 
in the text or by referral to other texts and then is asked to proceed 

further with his plans for therapy. If he chooses surgery, he is asked on 

/ 

Frame 13 to select from a half dozen possibilities the operation which seems 
to him most appropriate. His choices will reflect the adequacy of his 
diagnostic workup and his knowledge of the patient 9 e disease and previous 
treatment. 

You will note that this lower case is the same length as the upper case 
that we referred to previously. It makes much more extensive demands on the 
student's knowledge, however. In our opinion, to cover in a linear text all 
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the information that chi> one caw prewntation demand! of Che student 
would require a text of several hundred frames. In our linear text, we 
made no attempt to cover the content of a programmed case such as this. 

In our opinion, linear programming is not an efficient way to teach 
clinical trouble shooting. For this purpose, the branching technique seems 
more appropriate, less time consuming and certainly much easier to write. 
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APPENDIX C 



LEXTER FROM DOCTOR SCHUMACHER WITH DESCRIPTION OF 
SPECIAL NATIONAL BOARD OF MEDICAL EXAMINERS, EXAMINATIONS A AND B. 
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National Board op Medical Examiners 



»CLNttDtft*.trmUiAlii •••UAHM B. 

133 SOUTH 30TM STREET 



PHILADELPHIA A 



-OHN P HUBBARD, MO 
IKKCUTIV ( imiCTH 
EOITHE J LEV1T. M O 
ABBiRTANT BIRICTOR 

Charles p bchumachir. ph o 
•imctor »r tcbtins 
PAUL H KELLEY. JR , PH O 

ABBIRTANT BlRSCTOR mf TSBTINB 
CHRISTIAN J LAMSCRTSEN. M O 
•CCRCTaRT part I 
HAOLEYL.CONN. JR. MO 
SCCR1TAR V PART II 
TRUMANS SCHNABEL, JR . M O 

STCRlTAAV PART III 
■ VIPETTS CLWOOD 



carl a mover, m o 

PAIllBINT 



JAMES M PAULKNER. M D, 
VICE PMAISKNT 



EXECUTIVE COMMITTEE 
JAMES M FaULKNIR, M l. 
c J SLASPIL. M •. 

J O MCCARTHY, MS 
WILLIAMS, MISOLETON, M S 
CARLA. MOVER, M 0. 

JOHN PARKS. M O 
RICHARD H VOUNS. M O 



IRITtlB 



August 7, 1963 



CABLE ASMIBB NATSORD 



P. L. Wild*, M.D. 

Medical College of Georgia 
Department of Obstetrics and Gynecology 
Augusta 9 Georgia 



Dear Dr. Wilds: 



We have received your test msterlal and are proceeding with the 
printing ot teat booklets and answer aheata. We plan to run two hun- 
dred copies of each examination and have these in your hands by September 



In constructing these teata the following procedure! were followed 
to arrive at two equated examine tione; 



. lM h wert . dr '™ l Tom the National Board pool on Ob-Gyn neo- 

2? ■“* All had previoualy been ueed in National Board examination. end 

had been analyzed to determine their performance with National Board candidate!. 



, were classified into nine aub-categoriee by youraelf according 
*P* clflc ■ubject-matter arecs measured. Preliminary drafts of the 
two teat foriM were conetructed to have approximately equal numbera of item 

k AU dUpll "* te lte ~ «^hin . given te.t form were 
cither shifted to the other test or eliminated! 



, Th# Preliminary teat drafts were analyzed to determine their evereee 
diecrimination indices according to prtvious ptrformance with 
candidates. Soma items were ehifted from one teat form to 
another in order to make the two teata as equivalent as possible with reapect 

wh * nev,r * n «« ™ te.t,it w « 

teat? 0 d Wlth *" lt#m ln th# ,ub J ect “ m * tt,r mub-cetegory from the other 



fl 2? 1 5° rn *.° f ***• examination each contain one hundred and 
eight items. Ihe distribution of these items according to subject-matter 
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sub-categories are as follows: 

Humber of Items 



Category Form A Fora B 



1. 


Ovary and tube 


30 


25 


2. 


Sarcomas 


4 


3 


3. 


Fibroids 


6 


5 


4. 


Endometriosis 


17 


13 


5. 


Endometrium 


12 


10 


6. 


Endometrium and Cervix 


10 


12 


7. 


Cervix 


13 


27 


8. 


Vagina- vulva 


11 


8 


9. 


Choriocarcinoma 


5 


5 






108 


108 



The average difficulty levels of the final tests are: Form A .72, 

Form B .77. The average discrimination index (biserial correlation) of 
each form is: Form A .20, Form B .19. 

From these data it appears that the two tests, as a whole, are 
r ** ,on *bly equivalent. Zt should be noted, however, that no attempt was 
made to equate these tests within individual subject-matter sub-categories. 
Therefore, while it would be possible to obtain sub-sebres for each category, 
such sub-scores might or might not be equivalent in terms of difficulty or 
discrimination. 

I hope this provides the information you need regarding the method by 
which the tests were equated. Please let me know if you have further 
questions about this procedure. 



Sincerely, 



Charles F. Schumacher, Ph.D. 
Director of Testing 



CFS:aj 
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INSTRUCTIONS TO JUDGES (ORAL EXAMINERS) 
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"Effectlvenesa of a Programed Text In Teaching 
Gynecologic Oncology to Junior Medical Studenta" 



Ina true t Ion to Judaea (Oral Examiner.) 

ORIENTATION 

During each examining week (one in January, the other in Hay) you will be aaked 
to teat a group of abvut 45 junior medical atudenta in gynecologic oncology. Iheac 
atudenta will have either completed or nearly completed a nine week clerkehlp In 
obatetrlca and gynecology. In about equal number* the atudenta will have been taught 
the detection, dlagnoala, and aumagement of gynecologic tumor* by one of two aiethoda: 

(a) a program of lecturea by an outatandlng lecturer 

(b) an experimental programmed text 

Studenta from both groupa, in a acrambled order, will meet with the two examlnera 
by appointment. During the oral examination, the atudent may volunteer Information 
which will permit the examlnera to determine whether he la from the control (lecture) 
group or from the experimental (programmed text) group. The examlnera, however, are 
apeclflcally requeated not to try to determine from which group the atudent cornea and 
to attempt, lnaofar aa the atudent will permit them, to evaluate him without a 
knowledge of how he waa taught. lnaofar aa pOaalblc, we would like to withhold thla 
Information from the examiners till after the last examination In the group. This Is 
£HiZ information about the project which the examiners should not know. 

*• * guide to the acope of the project you will be aupplled in advance with a copy 
ot Objective* of the Courae." Thla la an outline of the "content” of both the 
lecture courae and the programmed text. The purpoae of the oral examination*, however, 
1* to evaluate the application of thla content to problem* of patient care, rather 
than almply to evaluate the atudent 'a knowledge of the content Itaelf. The knowledge 
of content la being evaluated aeparately by written examination* apecially prepared 
by the National Board of Medical Examiners* In the course of these examinations the 
atudent will anawer almoat all Item* from the National Board'* pool of queatlona in 
gynecologic oncology. Coplea of theae examination* will be made available to you If 
you wlah. 

Aa a guide to the type of queatlon we would like you to aak the atudent, we have 
prepared a dozen or ao aample queatlona which you may uac, not uae, or vary aa you 
pleaae. Each of theae queatlona la "open ended" and depending on what further Informa- 
tion la aupplled to the atudent, could lead to a variety of acceptable anawera. 



PROCEDURE 

1. Studenta will be given appointment* with the examining team at 45 minute Interval*. 
For each atudent the examining team will be aupplled with five IBM grade card*. 
Theae five card* will be marked aa follow*: 

(1) (atudent name) t "application," examiner "A" 

(2) (atudent name), "content," examiner "A" 

(3) (atudent name), "application," examiner "B" 

(4) (atudent name), "content," examiner "B" 

(5) (atudent name), "application." final grade 




228 



The«e five cards are to be completed by the examining team at the conclusion of 
each examination. 

2. We would like each oral examination to last 35 to 40 minutes. One examiner may 
aak queationa and carry on a diacuaaion with the atudent while the other liatena. 
How frequently the examinera rotate the queationing and liatcning rcaponaibility 
ia left entirely to the diacration of the examincra. 

3. At the conclusion of the examination the studant Is dismissed. The examiners 
should then Immediately, without consulting each other, write down their Indivi- 
dual grades for their estimates of the student's proficiency In "content" of 
gynecologic oncology and his proficiency In "application" of this content to 
problems of patient cara. This uses up four of the five cards for that student. 

4. After the two examiners have recorded their Individual grades, they should then 
discuss the studant s performance, If any discussion Is necessary, and arrive by 
consents or mutual agreement at a "final grade" which represents their Joint 
opinion of the student s proficiency In "application". The final pooled grade will 
be the criterion score for assessing "application" In this study. The indepen- 
dent grade ■ of judgai A and B on application and content will be used for 
statistical studies of the reliability and validity of this type of examination 

as compared with other methods of student evaluation used In this project. 

OTHER INFORMATION 



Examiners will be kept as fully Informed as possible by Quarterly progress 
reports and results of examinations (including thair own) as soon as they become 
available throughout the course of this project. 
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DESCRIPTIVE CATALOG Or PROBLEM-SOLVING TESTS 

I. EIGHT TESTS EMPHASIZING DIAGNOSTIC SKILLS 
1 • 

Presenting Problem: Post -menopausal vaginal bleeding 

Diagnostic Process: 

relevant information concealed by tab-item format in 

23 History Items: diabetes, high blood pressure, past 

history of syphilis, previous breast surgery 
for cancer. 

40 Physical Examination Items: Surgical absence of breast, ul- 

cerative lesion of vagina 

33 Diagnostic Tests and Procedures: biopsy of vaginal lesion, adeno- 

carcinoma; x-ray, evidence of 
wide-spread metastatic disease. 

Diagnosis; several appropriate choices in 50 options 

Therapy: several appropriate choices in 44 options 

2. Test A 1 

Presenting Problem: Same as A 

Diagnostic Process: 

relevant information concealed by tab-item format in: 

23 History Item*: diabetes, obesity 
40 Physical Examination Items: obesity 

33 Diagnostic Tests and Procedures: Vaginal cytology: suspicious 

Fractional D&C, cndocervlx: 
adenocarcinoma 
endometrium: 
estrogenic hyperplasia 

Diagnosis: two options, including several appropriate choices 

Therapy: 44 options, including several appropriate choices 
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3. Teat A 

Presenting Problem: Sam as A 

Diagnostic Pro r 

relevant information concealed by tab-item format in: 

23 History Items: Same as A 1 

40 Physical Examination Items: Same as A 1 

33 Diagnostic Tests and Procedures: Vaginal cytology: negative 

Fractional D&C, endocervix: endocervicil 

tissue 

endometrium: adeno* 

acanthoma 

Diagnosis: ssveral appropriate choices in 50 options 

Therapy: aaveral appropriate choices in 44 options 

4. Test C 

Presenting Problem: Abdominal pain and distention 
Diagnostic Procsss: 

relsvant information concealed by tab-item format 1?: 

23 History Items: non-contributory information in 23 categories 

40 Physical Examination Items: findings suggestive of pelvic mass and 

ascites 

37.Dlagnostlc Tests and Procedures: peritoneal fluid, cytologic changes 

suggestive of malignancy 

Diagnosis: ssveral appropriate choices in 38 options 

Therapy: one appropriate sequenced treatment pattern Included in 50 options* 
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5. Teat p 



Pteientlns Problem; Sudden onset of pain in right lower quadrant 
Diagnostic Process; 

relevant information concealed by tab-item format in 

23 History Items; symptoms suggestive of acute appendicitis 

40 Physical Examination Items: findings suggestive of acute surgical 

abdomen with right lower quadrant pelvic 



36 Diagnostic Tests and Procedure Items: Sample of peritoneal fluid, 

evidence of intraperitoneal 
bleeding 

Diagnosis: 8 options offered must be ranked in order of probability. 

Therapy: One appropriate and several inappropriate options favored. 

6. Teat D* 

Presenting Problem: Suudcn onset of pain in right lower quadrant 

Diagnostic Process: 

relevant information concealed by tab-itsm format in 

23 History Options: minor alterations from D not affecting diagnostic 

problem 

40 Physical Examination Options: minor alterations froei D not affecting 

diagnostic problem 

36 Diagnostic Tests and Procedure Options: several alterations from D 

suggesting infectious process; 
sample of peritoneal fluid, pus 

Diagnosis: 8 options offered must be ranked in order of probability 

Therapy: Ono appropriate and several inappropriate options offered. 
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7. Teat E 



Presenting Problem: Routine pre-employment physical 
Diagnostic Process: 

relevant information concealed by tab-item format in 
23 History Options: previous occupation, prostitute 

40 Physical Examination Options: small ulcer on vulva 

33 Diagnostic Tests and Procedure Items: VDRL, reactive, titer 1:64; 

Darkfield Examination of lymph from 
ulcer: negative for spirochetes 
Biopsy of ulcer: invasive squamous 
cell carcinoma 

Diagnosis: 37 options offered, 3 correct ones must be chosen 

Therapy: Any of several patterns in 33 options 



8. T est E* 

Present: Problem: Same as E 

Diagnostic Process: 

relevant information concealed by tab-item format in 

23 History Options: minor changes from E, not of diagnostic significance 

40 Physical Examination Options: changes from E not of diagnostic 

significance 

36 Diagnostic Tests and Procedure Options: VDRL, non-reactive 

Darkfield examination, negative; 
Smear of Lesion: positive for 
Donovan bodies 

Biopsy of ulcer: granulomatous 
lesion 

Diagnosis: of 37 options, one is correct 

Therapy: 33 options offered, several are acceptable. 




II. 



TEST EMPHASIZING THERAPEUTIC figm. 



The following tent was primarily designed to measure skill in 
a previously defined (diagnosed) clinical problem: 



management of 



Test B, First Part of Teat. 



Presenting Problem: 30-year-old primigravida at 26 weeks gestation requiring 

prenatal care. 



Diagnostic Process: 



relevant information concealed by abbreviated 
workup; suspicious vaginal cytology. 



(8 option) diagnostic 



Management Options Offered: 



Wait till patient*s next visit for routine antepartal care and inform her 
that the reports were satisfactory. 

CaU the patient back to your office, inform her of the results and repeat 
the Pap smear. v 

CaU the patient back to your office, inform her of the results and perform 

a Schiller test and obtain cervical punch biopsies from any non-staining 
areas ° 

Admit the patient to the hospital for cervical conization and undocervical 
curettage. 

Empty the uterus by hysterotomy and refer patient to a radiologist for 
therapy. 

Call in a cancer specialist to handle the problem. 

None of the above options 



The student is then given appropriate information leading him to further 
management options. 

The student is then asked to consider in sequence each of the following diagnoses: 

a. Cervix showing decidual reaction compatible with pregnancy. 

Endocervical tissue showing squamous metaplasia with minimal atypia. 

b. Atypical squamous cell metaplasia (dysplasia of cervix and endocervix) 

c. Cervix with intraepithelial (pre- invasive) squamous cell carcinoma of 
the cervix with invasion of endocervical glands. 

d. Invasive squamous cell carcinoma of the cervix extending to the margins 
of the specimen submitted, disease staged clinically as la 

For these diagnoses, he is asked to consider his management if the diagnosis 
were made on the basis of a specimen obtained either by 

1. cervical punch biopsy, or by 

2. conization and endocervical curettage. 

and for each, to make the best choice from the following options: 
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Perform a cone biopay of the cervix and curet the endocervix in the 
third trimeater of the pregnancy 

Deliver vaginal ly at term, re-evaluate the cervix postpartum 

Deliver by Caeaarean eection at term, then atart definitive treatment of 
cervical leaion 

Let pregnancy continue to term, then deliver by Caeaarean hyaterectomy 

Interrupt pregnancy by hysterotomy, then treat the cervical leaion by 
appropriate surgery. 

Interrupt pregnancy by hyaterotomy, then treat the cervical leaion with 
a full courae of x-ray therapy and intracavitary radium. 

Ignore the pregnancy, perform a radical hyaterectomy with pelvic lymphaden- 
ectomy aa aoon aa poaaible. 

Ignore the pregnancy, treat the patient with a full courae of intracavitary 
radium and external x-ray therapy. 

Let pregnancy continue to fetal viability, deliver the fetua by Caeaarean 
aection, then treat the cervical leaion by appropriate therapy. 

Let pregnancy continue to fetal viability, then induce labor, deliver 
infant vaginally and treat cervical leaion with radiation therapy. 

Refer the patient to a apecialiat in oncology or radiology for further care. 



Second Part of Teat 

Prcaenting Problem: Patient at 6 week* poatpartum requiring further management. 

Student ia required to conaider each of the following diagnoaea in aequence. 

1. Pre-invaaive aquamoua cell carcinoma of the cervix, eatabliahed by cone 
biopay in mid -pregnancy. 

2. Pre-invaaive aquamoua cell carcinoma of the cervix, eatabliahed by 
conization and fractional D&C at aix weak a poatpartum. 

3. Invaaive aquamoua ceil carcinoma of the cervix extending to the margina of 
biopay apecimen aubmitted (cone or punch); diaeaae ataged clinically aa 
Stage la. 

For each of theae, the atudent ia aaked to aelect appropriate therapy from the 
following optiona: 

Perform a fractional D&C and conization of the cervix 

Perform a total hyaterectomy 

Perform a radical hyaterectoaty with pelvic lymph node diaaection 

Irradiate the patient uaing internal radium aourcea and external x-ray to give 
a total doaage of 16,000r at Point A and 8,000r at Point B in 6 weeka 

Irradiate the patient uaing external x-ray and internal radium aourcea to give 
a total doaage of 8,000r at Point A and 6,000r at Point B in 6 weaka 

Refer the patient to a apecialiat for further core. 
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Thi» "clinical problem aolving taat ,f conaiata o£ a caat preaentation in 
a format which is deaigned to taat your ability and judgement in tha dlagnoala and 
treatment of tha patient 'a disorder. You ara provided with a taat booklet and 
a apaclal anawar aheet* Ihe tvo Must ba uaad togathar. Tha taat la dividad into 
thraa part a. 

Fart I* Coilacting inforaatlon about tha patient* 

Part II. Daflnlng tha patient f a dlagnoala* 

Part III* Spaclfylng your plan of treatment for tha patiant* 



For aach part of tha teat, you will uaa a dlffarant portion of tha anawar 
ahaat in a dlffarant way. You mey work through tha taat or inapact any part of 
it in any ordar you ch ooae, but plaaaa ba caraful to follow tha apaclal 
lnatructlona for aach aactlon* 



Tha taat beglna on PAGE 2* 



Praparad by; P. L. Wilda» M.D. and Virginia Zachert* Ph.D. 
©apartment of Obatatrlca and Gynacology 
Madlcal Collaga of Georgia 
Auguata, Garrgla 



Copyright (c) 1965 » Madlcal Collage of Georgia 



September 1965 (mho) 
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CASE PRESENTATION 



0 



S e ing she has noted since her Menopause two years ago at age '*8. 



0 



0 






0 



DIRECTORY 



Part I. 

Parc II. 
Parc III. 



Collaccing Inforaicion, Inacrucciona. 

A. HilCory “ — — 

B. Phyaical Examination 

C. Diagnoacic Scudiaa and Procedurea, Inacrucciona. 

Lirt* of SCudiaa and Procedurea • 

Defining Diagnoaia, Inacrucciona. 

Liaca of Diagnoaia 
Specifying Treacaienc, Inacrucciona. 

Liaca of Treacmenca 



PAGES 3-7 
PAGE 9 
PAGE 11 
PAGES 12, 13 
PAGE 15 
PAGE 16 
PAGE 17 
PAGES 20, 22 
PAGES 21, 23 



Pleaae go on co page 3. 



a 

o 

o 

D 

Q 



0 

0102 

D 




D 



0 

id 

n 

U 



9/28/65 



239 



FACE 3 
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Inatructlona for Collecting Information 



|g5JgSi« 9. 11. *nd 15 of thla tut, labelled MORE HISTORY, PHYSICAL 
EXAMINATION, and DIAGNOSTIC STUDIES AND PROCEDURES, arc deaigned to provide 
you with information about the patient, but they give you only the information 
you aafc for. 



D 

n 



n 

u 



ISSSS^Sl SjUl Booklet e The right-hend <pdd- numbered) side of each of the three 
pages contains a list of categories of parts of the history, parts of the 
physical examination, and various tests and procedures. Each item is followed 

thi rl «^ t-,h *nd margin (HISTORY items begin with 150. PHYSICAL 
EXAMINATION 250, TESTS AND PROCEDURES with 350, etc.) 

j 

Jjjg gS&tifr °P* n th * M,t booklet to page 9, 11, or 15 and lnapect the right- 
narti aide of the page. 



On the left-hand (even numbered) pagea 8, 10, and 14, you will find that 
there ia a column of numbera followed by a acraabled Hat of conflicting 
atatenanta about die patient. Each atatement la preceded by a numbe r 
(beginning with 100 for hlatory, 200 for phyalcal examination, 300 tor teata 
and procedurea, etc.). Some atatementa in the llat are directly applicable 
to your patient, othera are Irrelevant or bogua. The anawer aheat la the 
key which tella you which information la applicable to the patient. 



Sxerciae 2. Open the ceat booklet to "information" pagea 8, 10, or 14 and 
lnapect the left-hand aide of the page, then return to PAGE 5. 



n 

u 



After you have completed excrclaea 1 and 2, go on to FACE 5. 
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000. Read instruction* on PAGE 5 FIRST 

001. B* *ur* you understand instructions on PAGE 5 

002. Be sure you understand instructions on PAGE 5 

003. Erase item 036 on answer sheet, and follow Instructions on this pace for 
the number you erase. 

004. Be sure you understand Instructions on PAGE 5. 

005. Be sure you understand instructions on PAGE 5. 

006. Be sure you understand instructions on PAGE 5. 

007. Be sure you understand instructions on PAGE 5. 

008. Be sure you understand instructions on PAGE 5. 

009. Be sure you understand instructions on PAGE 5. 

010. proceed to Instruction #2. 

011. Be sura you understend instructions on PAGE 5. 

012. Be sure you understand instructions on PAGE 5. 

013. Be sure you understand instructions on PAGE 5, 

°J« ®* ,ur * understand instructions on PAGE 5. 

015. Be sure you understate instructions on PAGE 5. 

016. Be sure you understand instructions on PAGE 5. 

017. Be sure you understand instructions on PAGE 5. 

018. This I* Just to practice erasing lumbers. 

019. Be sure you understand instructions on PAGE 5. 

020. Be sure you understand Instructions on PAGE 5. 

021. Be sure you understend Instructions on PAGE 5. 

022. Proceed to PAGE 6. 

023. Be sure you understend instructions on PACE 5. 

024. Be sure you understand instructions on PAGE 5. 

025. Be sure you understand instructions on PAGE 5. 

026. Be sura you understand instructions on PACE 5. 

027. Proceed to Instruction #3 on PAGE 5. 

028. Be sure you understand Instructions on PAGE 5. 

029. This is Just to practice erasing nuebers. 

030. Be sure you understand instructions on PAGE 5. 

031. This is Just to practice erasing numbers. 

032. Be sure you understand instructions on PAGE 5. 

033. Be sure you understand instructions on PAGE 5. 

034. Be sure you understand instructions on PAGE 5. 

035. Be sure you understand ins true -ions on PAGE 5. 

036. Be sure you understand instructions on PAGE 5. 

037. Be sure you understand instructions on PAGE 5. 

°3». Be sure you understand instructions on PAGE 5. 

039. Be sure you understand instructions on PAGE 5. 

040. Be sure you understand instructions on PAGE 5. 

041. Be sure you understand instructions on PAGE 5. 

042. Be sure you understand instructions on PAGE 5. 

043. Be sure you understand instructions on PAGE 5. 

044. Be sure you understand instructions on PAGE 5. 

045. Be sure you understand Instructions on PAGE 5. 

046. Be sure you understand instructions on PAGE 5. 

047. Be sure you understand instructions on PAGE 5. 

048. Be sure you understand instructions on PAGE 5, 

049. Be sure you understend instructions on PAGE 5. 
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Aniwer She£t, The answer sheet consists of ten columns numbered from 0 to 9. The 
columns are made up of numbers in numerical order, ranging from 050 to 099 in 
column 0 to 950 to 999 in Column 9, Each of these numbers corresponds to the 
item with the same number on the r ight-hand page of the test booklet. To the 
right of each of the first four colurans"on the answer sheat, there is a stripe 
of erasable ink. Beneath this stripe there is a column of numbers in scrambled 
order. These concealed numbers correspond to numbered items on the left-hand 
(even numbered pages) of the test booklet, fo obtain information about your patient, 
you must erase the proper areas in each stripe of the answer page, read the 
concealed numbers, then read the items with the corresponding numbers on the left- 
hand pages of the test booklet and be guided by the information you arc given. 



Exercise 3, The first coIurti, Column 0, is for practice. 

Instruction #1, On the answer sheet, in Column 0 please erase item 050. 
When you have done so, Item 050 on the answer sheet should look like this: 

050 003 

Now look at Page 4 (opposite) and follow the instructions given for Item 003. 



Instruction #2 . If you followed the instructions for Item 003 on page 4, the 
top of Column 0 of the answer sheet should now look like this: 

050 

051 

052 

053 

054 

055 

056 010 

057 

058 

Now please practice erasing items 051, 052, 053, and 054, 





Initr uc^lon_#3 > If you have completed Instruction #2, tho top of Column 0 
should now look like this: 

050 003 

051 029 

052 018 

053 031 

054 027 

055 ■§ 

056 010 

057 

058 

Now please erase Item #055, and follow instructions. 
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^corinj. The part* of thii teat dialing with hiatory and phyaical examination 
have two requirement* which muat be completed in thia order: 

FIRST: You muat eraae thi proper itema to get the information you need« 

SECOND: You muat aaaign to proper categoriea all the eraacd item* in 
theae parte of the teat. ~ 

You will receive a acore (poaitive or negative) for each numbered item in 
the teat booklet whether you mark it or not. Pleaac do not ekip any items but 
conaider each one carefully. 

FIRST: A. Eraae. all itama which fall into theae two categoriea: 

Indicated, item*. Theae are one* where the collection of infor- 
mation ia directly related to the patient'* problem aa it haa 
preaented itaelf to you. For example, in a patient with a 
hiatory of hypirtenaive diacaae, determining the patient'* 
blood preaaure would be clearly an "indicated" Item. 

2< Routine itema uaed for acrcening, ruling out complication*, 
or adding to uaeful general information about the patient. 

B. Do not eraae any of the following itema: 

ill&l&U, •&£££• Theae are itema which have no bearing, direct 
or indirect, on the patient'* problem and are conaidercd 
valueleaa even for acrcening or aurvey purpoaca. 



Go on to PAGE 7. 
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SECOND: Aaalgning Itm to Catetorlee: 

A. Eraeed I tem, 

1. If you com idee that the item waa clearly "indicated" by the nature 
of the patient’* problem, make a mark to the rltht of the eraaed 
number in the column Indicated, like thia: 

199 143'—’ 

2. If you conaidered that the item you eraaed waa "routine 1 1 and 
waa uaeful only for acreening or for general Information or 
perhapa Jmt to aatiafy your curioaity, make no mark, Juat leave 
the number aa it atanda, like thia: 

199 143— 

B. Unerased Item* . 

1. If the item you didn't eraae waa conaidered uaeleaa but harmleaa, 
make no mark, leave the item aa it atanda, like thia: 

199 Mh" 

C. Changing Your Mind . 

Once the number on the anawer aheet haa been eraaed, it can't be 
"recovered", ao don't try to. You may eraae or add pencil marka 
aa you aee fit. 



Inatructlon : Proceed to conalder alj. item* on PAGES 9 and 11. 



Reminde r: Be aure to consider each item. Remember that all itemi are acored, 

even the onea you leave uneraaed (the acore mty be poaltive or 
negative, depending on the item.) 
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INFORMATION 



100. Sometimes incontinent 

101. Chronic alcoholic 
102 • Living and well 

103. None 

104. Haa apartoant in own houae 

105. Uaea Ex-lax occaaionally 

106. Alwaya •'nervous.** 

107. Collage graduate 

108. Frequent backache 

109. Perioda 12x30x5, were prolonged and irregular for 3 yeara before 
menopauae at 48. 

110. None 

111. No operationa 

112. Teetotaler, on 1800 cal. diet 

113. Waara glaaaea for reading 

114. Aaymptomatic 

115. Diabete. 10 year, duration; .yphili. 15 year, ago, adequately treated. 
Brea.t cancer 8 year, ago, treated by .urgery, 

116. Takea 1 go. Tolbutamide dally 

117. Appendectomy at 23, left mastectomy at age 40. 

118. High achool 

119. Uaual childhood dlaeaaea only. 

120. None 

121. Haan*t felt well for yeara. 

122. None 

123. Haa bean taking "female hormone 1 * pille for yeara for **the change. *' 

124. Occasional frequency, no dyaurla. 

125. None 

126. Live a with huaband 57. 

127. Haa dlabetea & high blood pressure. 

128. No information available 

129. All in Europe 

130. Had cancer of (?) at age 46, and i a separated from apouac. 

131. Runa boarding houae 

132. No recent change 

133. Spouae died 4 yeara ago of Tbc, 

134. None 

135. Croaa hematuria (one day epiaode) 2 c’ontha ago. 

136. Regular and aatiafactory (friend renta room from her) but haa 
had poatcoital bleeding for 6 weeka. 

137. Severe 

138. Severe ahortneaa of breath and minimal excretion 

139. None 

140. Sometimes haa palpitationa. 

141. Frequent occipital headaches. 

142. Died of cancer of the womb 

143. Eats **what ahe pleases," moatly carbohydrates. 

144. Still bleeding 

145. Suffers from hemorrhoids 

146. None noted 

147. Patient refuses to answer 

148. You can't get here from there. 

149. Living and well. 
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MORE HISTORY 

You <n«y a««une that the chief Complaint and Pre.ent Xllneaa a. given are comolete 
n correct. For additional information please aelect AS MANY of the item* below 
aa intereat you, oraae the code numbera of theae item* in the proper column of the 

tron«n5?L t, « thCn f n ind *»*"™*™ with the corresponding^* nSbS on 
c oppoaite page* On the anawer aheet, make your craaurea in COLUMN 1 , 
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Paat Medical Hiatory 



Family Hiatory 



Social Hiatory 



Syatem Review 



Illneaaea 


150 


Injuriea 


151 


Operationa 


152 


Pregnanciea, 


153 


Father " 


154 


Mother 


155 


Siblinga 


156 


Other a 


157 


Schooling 


1*8 


Occupation 


139 


Hone Environment 




Marital aituation 


161 


Sex lift ~ — — ^ 


162 


Habita 


163 


Drug, and Medicine, “ “ 


164 


General (wgt., fever, weakneaa, 
HEENT 


etc.) 165 
166 


CVR 


167 


GI 


168 


GU " 


169 


GY 5 — ~ 


170 


NP “ 


171 


Muaculoakelctal 


172 



■"* M ™ 150 * 172 " th * ■“» 



Routine or acreening item 
Indicated, caaentlal item 
Uaeleaa but harmlcaa item 



199 143"»“ (no mark) 

199 143*»* (mark tr right) 
199 (no mark) 



When you have completed your marking, proceed to PAGE II. 



0109 



( 

* 

► 



o 







246 



PAGE 10 



9/28/65 



INFORMATION 
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200. Not noted 

201. All present and equal 

202. None palpable, 

203. 5*0", 190 lbs. 

204. Well-formed 

205. Not enlarged 

206. Not enlarged 

207. Fungating exophytic lesion 

208. Intact 

209. Atrophic 

210. Grade II changes, capillary ntLcroaneurisms. 

211. Enlarged to level of umbilicus. 

212. No abnormalities noted 

213. Supple 

214. 2 cm. ulcer on posterior wall at hymenal ring. 

215. 37°, 80, 18, 180/112 

216. Obese 

217. Not felt 

218. Moist 

219. Old third degree laceration 

220. Normal 

221 d Not palpable 

222. Unremarkable 

223. Right normal, papilledema of left disc. 

224. Undis tended 

225. Physiologic 

226. Old mastectomy scar on left; right negative. No nodes. 

227. Well-formed 

228. Well-formed 

229. Well-developed, obese W # F. 

230. Unobstructed 

231. Not enlarged 

232. Noth* ^ abnormal 

233. No c.onormalities 

234. Atrophic 

235. Distended, typanitic with hyperactive bowel sounds 

236. Normal size, no mu r mu rs 

237. Within normal limits 

238. Tremendously obese with old laparotomy scar 

239. Eeft drum perforated 

240. Mid line 

241. Not noted 

242. Intact 

243. Moderate enlargement, totally irregular rhythm, no murmurs 

244. No abnormalities noted. 

245. Confirms pelvic findings. 

246. Atrophic 

247. Examination unsatisfactory 

248. Pap smear taken, see report 

249. Not enlarged, mid-position. 
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GENERAL PHYSICAL EXAMINATION 



of 6 thp !nl eCt A h ° f the items below 88 you wi8h to examine . In the proper column 

with the Shee !jj er3S j the C ° de numbers of thes ® itema, and look up the findings 

with the corresponding code numbers on the opposite page. 88 



On the answer sheet, make your erasures in COLUMN 2. 



TPR, BP 


250 


Hgt., Wg t • 


251 


General description 


252 


Skin 


253 


Lymphatics 


254 


Head and face 


255 


Hair 


256 


Eyes 


257 


Ears 


258 


Nose 


259 


Mouth, teeth, throat 


260 


Neck 


261 


Trachea 


262 


Thyroid 


263 


Vessels 


264 


Chest 


265 


Breasts and axillae 


266 


Heart 


267 


Lungs 


268 


Abdomen 


269 


Liver, spleen, kidneys 


270 


Masses 


271 


Tenderness 


272 


Pelvic examination 


273 


Hair distribution 


274 


Ext. genitalia 


275 


SUB glands 


276 


Introitus and perineum 


277 


Vagina 


278 


Cervix 


279 


Uterus 


280 


Adnexa 


281 


Rectal 


282 


Sphincter 


283 


Masses 


284 


Back 


285 


Extremities 


286 


Pulses 


287 


Deep tendon reflexes 


288 


Neurological 


289 



to e the°fo??o'ing n, cide t t d y ° Ur eraSUreS * m8rk items 250 ‘ 289 -« sheet according 



Routine or screening item 
Indicated, essential item 
Useless, but harmless item 



199 143*** 
199 143— 
199 



After you have completed your marking, proceed to PAGE 12, 



(no mark) 

(nup-k to right) 
(no mark) 
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INSTRUCTIONS FOR DIAGNOSTIC TESTS AND PROCEDURES 



Scoring. The part of this test dealing with diagnostic tests and procedures is 
similar to the part dealing with history and physical examination, but has an 
additional requirement. 

FIRST. You must erase the proper items to get the information you need 

SECOND. You must assign to proper categories all items in this part of the 
test, both erase d and unerased (This is the new requirement). 

You will receive a score (positive or negative) for each numbered item in this 
part of the test whether you mark it or not. Please do not skip any items but 
consider each one carefully. 



FIRST* A. Erase all item3 which fall into these two categories; 

Indicated Iteni3 * These are ones where the collection of 
information from diagnostic tests or procedures is directly 
related to the patient's problem as it has presented itself 
to you. 

2* Routine Items * These are items used for screening or survey or 
for ruling out complications, not directly related to the 
patient's primary illness. 

B. Do not erase any of the following items. 

Use, leas items . These are diagnostic tests and procedures 
which have no bearing, direct or indirect, on the patient's 
problem but are essentially harmless. They may, however, cost the 
patient time, money, and minor discomfort or anxiety. 

2* Contraindicated items . These are tests or procedures which 
subject the patient to unnecessary and unjustifiable risks, 
anxiety, pain, or discomfort. 



Go on to PAGE 13. 



* 
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SECOND. Assigning Items to Categories . 

A. Erased Items. 

1. If you consider that the item was clearly Indicated by the nature 
of the patient's problem, make a mark to the right of the erased 
number in the column inuicated, like this: 

399^**319— 

2* If you considered that the item you erased was "routine" and 
was useful for only screening or for general information, or 
perhaps just to satisfy your curiosity, make no mark, just 
leave the number as it stands like this: 

39 9-^-3 19=** 

B. Unerased Items. 

1« If the item you did not erase was considered useless but 

harmless make no mark, simply leave the item as it stands like 
this: 

399«*4HK** 

2. If you decide not to erase a number because it seems contra- 
indicated, harmful and not in the patient's interest, make a 
mark in the space provided to the left of the unerased number 
like this: 

0. Changing Categories , Occasionally, there, will be a diagnostic tast 
or procedures which at first appear to be "contra-indicated" and should 
be marked as such. Later, when you gain additional information about 
the patient, this previously "contra-indicated" item now becomes 
"indicated." This can be shown by erasing the number and making 
marks on both sides of it like this: 

399— 319- 



Instructions: Proceed to consider ill items on PAGE 15. 

Reminder : Be sure to consider each item. Remember that all items are scored, 

even the ones you leave both unerased and unmarked. 
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INFORMATION 

300* Negative 
301 • Not indicated 

302. Negative 

303. Non-reactive 

304. Negative 

305. Negative 

306. Negative 

307. Squamous call carcinoma, invasive 

308. Left ventricular hypertrophy 

309. No abnormalities 

310. Na 140, K 3.8, CL 98, C0 2 25 

311. Class I, atrophic smear 

312. Report not available 

313. Negative 

314. Class II, estrogen effect 

315* Class IV (positive) malignant cells present 

316. No abnormalities 

317. F 100, 1 hr. 220, 2 hr. 190, 3 hr. 140 

318. Hct. 36, WBC 8,000, differential normal 

319. Marked cardiac enlargement with hypertensive contour. Left brease 
shadow absent. 

320. Less than 6 7, retention at 45 minutes. 

321. No evidence of extension boyond tho uteru., no enlarged lymph node, 
or signs of peritoneal spread. 

322. 180 mg7. 

323# Aortic lymph nodes are enlarged, and on biopsy and frozen section 

they show adenocarcinoma. Metastases to liver are also palpable. No 
evidence of peritoneal spread. 

3i4. Specific gravity 1.010, pH 5*8 glucose 2+, acetone negative, albumin 
positive, microscopic t occasional WBC, 

325. Chronic cervicitis with squamous metaplasia 

326. Reactive, titer 1:64 

327. Report not available 

328. Chronic cervicitis with squamous metaplasia 

329. 0, Rh positive 

330. Negative 

331. 4 K-A units/100 ml. 

332. Negative 

333. Negative 

334. Findings: same as noted elsewhere. 

335. Patient dies on operating table of pulmonary edema. 

336. Negative film. Heart normal size. 

337. Endocarvlcal tissue 

338. F 80, 1 hr. 110, 2 hr. 68, 3 hr. 80 

339. 10 rnguft 

340. Negative 

341. Positive 

342. Chronic cervicitis 

343. Na 120, K 5.1, Cl 86, C0 2 11 (nto/1) 

344. Negative 2 * 9 

345. Scattered spherical ("snowball”) densities in both lung fields, minimal 
cardiac enlargement 

346. Negative 

347. Hematocrit 23, WBC 6,000, hypochromic, microcytic anemia. 

343. 40Z excretion in 15 minute «• 

349. Adenoacanthoma. 
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DIAGNOSTIC STUDIES AND PROCEDURES 



Please select AS MANY of the items below as you wish to examine. In the proper 
column of ».he answer sheet* erase the code numbers of these items, and look up the 
findings with the corresponding code numbers on the opposite page. 



Chemistries 

(blood, serum) 



Clinical & 

cytc-pathology 

serology 

Hematology 

Urine tests 
X-rays 



Procedures 



Diagnostic Surgery 



■newer ah.ee, make your erasure. In COLUMN 3. 
Aik. phosphatAse 


350 


Bilirubin, direct, Indirect 


351 


Glucoaa, 2 hr., poatprandlal 


352 


Electrolyte., Na, K, Cl, CO2 


353 


Urea Nltrdgen (BUM) 


354 


Stool for blood, OCP 


355 


Vaginal pap anear 


356 


VDRL 


357 


Blofld group, and Rh 


358 


CSC 


359 


Urinalyala, couple te 


360 


Abdomen ^ 


361 


Barium enema 


362 


Chest 


363 


Cholecystogram 


364 


GI aerl.a 


365 


Pelvla 


366 


Py.logram (IVPj ~~ 


367 


Skull 


368 


Spine 


369 


BSP 


370 


Cystoscopy 


371 


Darkfleld exam for T. Pallidum 


372 


Electrocardiogram 


373 


Examination under anesthesia 


374 


Frel test 


375 


Glucoie tolerance test 


376 


PPD or Tuberculin Test 


377 


Proc tos igmoidoscopy 


378 


PSP 


379 


Smears for Donovan bodies 


380 


SMara for H. Ducreyi 


381 


Biopay cervix (punch) 


382 


Blopay vagina 


383 


Coniratlon of cervix 


384 


DSC, endocervlx 


385 


DSC, endometrium 


386 


Exploratory laparotomy^ 


387 



After you have completed your erasures, mark items 350 to 387 on the asnwer sheet 
according to the following code; 



Routine or screening item 
Indicated, essential item 
Useless but harmless item 
Contraindicated, harmful item 



199—143— 

199—143— 

19 9 — m m — 

199— 1— 



After you have completed your marking, proceed to PAGE 16. 



(no mark) 

(mark to right) 
(no mark) 

(mark to left) 



i 

; 
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PART II 
YOUR DIAGNOSIS 
DESCRIPTION OF PART II 



Th^-tg g t _bookl# t tj This section of the test consists of lists of primary and 
secondary diagnoses which you are asksd to divide into thres categories: 

1« Diagnoses which have been excluded by your history, physical 
exam, or diagnostic tests and procedures. 

2. Diagnoses which were not excluded by your history, physical 
exam, or diagnostic tests and procedures* 

3. Diagnoses which were established or rated most likely by your 
history, physical examination and/or diagnostic tests and 
procedures. 

jh* answer^ sheet, • Column 4 of the answer sheet consists of a column 
of numbers corresponding to the code numbers of the listed diagnoses* 

Ihe column of numbers is followed by three columns of spaces in which 
you are to mark yo»*r answers with pencil as follows: 

1. Column 4 EXCLUDED is for diagnoses you have excluded. 

2* Column 4 NOT EXCLUDED Is for diagnoses you have not excluded* 

3* Column 5 ESTABLISHED is for estab lished or most likely 
diagnoses 

Instructions for PAGE 17- 

FIRST: From the list on the opposite page, select all the diagnoses which 

your workup of history and/or physical examination and/or diagnostic studies 
and procedures has permitted you to exclude from further consideration. Mark 
each of these in the Column 4 EXCLUDED of the answer sheet* at its proper number* 

SECOND: From the list on the opposite page, select al^ the diagnoses which you 
were unable to exclude by the choices of items of history, physical examination, 
and diagnostic tests and procedures which were available to you. Mark each of 
these in Column 4 NOT EXCLUDED of the answer sheet, at its corresponding number. 
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0 

0 

0 

D 

0 

0 



D 



0 

D 

D 

0 



LIST OF DIAGNOSES 
COLUMN 4 



Adenoaccnthoma, primary, of the endometrium 
Adenocarcinoma, primary, of cervix 



PAGE 17 



450 



Adenocarcinoma, primary of endometrium 


461 


Adenocarcinoma, primary of vagina 


466 


Adenocarcinoma, metastatic from primary in breast 


471 


Adenocarcinoma, metaatatic from primary in colon 


476 


Adenocarcinoma, metaatatic from primary in ovary 


481 


Carcinoma, aquemoua cell, of cervix 


485 


Chancroid 

Diabetes nwTiitus ‘ " — 1 


491 

492 


Granuloma inguinale - “ ‘ — 


493 

494 


Hypertensive vascular disease — - - ■ ■ — 


495 


iij(npnop«cnia venereum ^ — 


ruimonary cuoercuiosis, active 


497 


Pulmonary tuberculosis, inactive 


498 




499 


Instruct Iona: When you have completed thia page, proceed to PAGE 18. 



Q 



0117 
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i 

PAGE 18 



t 



! 

I 



INSTRUCTIONS FOR PAGE 19 



From your choice of the diagnoses which were not excluded by your workup, 
plesse indicate in the list on PAGE 19 the diagnose* which are definitely 
established or, of the choices given, moat likely* Of competing or 
conflicting diagnoses, there can be only ONE which is most likely* The 
patient may, however, have a number of unrelated conditions in addition* 

Mark each of your selection* in COLUMN 5 ESTABLISHED of the answer sheet, at 
its proper number* 






! 



i 



I 



s 

i 
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ESTABLISHED OR HDST LIKELY DIAGNOSES 
COLUW 5 



Adenoacanthoma, primary, of endometrium (stage unspecified) 


550 




State I 


551 


(specify stage if you can) 


Stage II 


552 


Stage III 


553 


Adenocarcinoma, primary of cervix (stags unspecified) 


Stage IV 


554 




555 




Stage 0 


556 




Stags I 


557 


(specify stage if you can) 


Stage II 


558 


Stags 111 


559 




Stag* IV 


560 


Adenocarcinoma* primary, of endometrium (stage unspecified). 


561 




Stage I 


562 


(specify stage if you can) 


Stage II 


563 


Stags III 


564 


Adenocarcinoma* primary Of vagina (stage unspecified) 


Stage IV 


565 




566 




Stage I 


557 


(specify stage if you can) 


Stags II 


568 


Stags III 


569 


Adenocarcinoma, metastatic from primary in breast 


State IV 


570 




571 




With spread to cervix 


572 


(specify spread if appropriate) 


With spread to lunts 


573 


With spread to ovaries 


574 




With spread to vatina 


575 


Adenocarcinoma* metastatic from primary in colon 




576 




With spread to cervix 


577 


(specify spread if appropriate) 


With spread to lungs 


578 


With spread to ovaries 


579 


Adenocarcinoma, metastatic from primary in ovary 


With spread to vagina 


580 




581 




With spread to cervix 


582 


(specify spread if appropriate) 


With spread to lungs 


583 


With spread to vagina 


584 


Carcinoma* squamous cell of cervix (stage unspecified) 




585 


State 0 


586 




Stage I 


587 


(specify stage if you can) 


Stage II 


. 588 


State III 


589 




Stage IV 


590 


Chancroid 




591 


Diabetes mellitus 




592 


Exogenous obesity 




593 


Granuloma inguinale 




594 


Hypertensive vascular disease 




595 


Lymptnpnthia venereum 




596 


Pulmonary tuberculosis, active 




597 


Pulmonary tuberculosis* inactive 




595 


Positive serology 




599 



Ins tru ction*: 



When you have completed this and all preceding pages in Parc II, 
proceed to PART III on PACE 20. 



0119 
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PART III 

YOUR PLAN OF TREATMENT 
DESCRIPTION OF PART III 



On the basis of the information which haa been .•’.ade available to you, you should 
not only be able to define your patient's proMniu, you ahould alao be able to 
outline a plan of managing thia patient'a major trincss. This part of the test 
is divided into two sections. 

1. Selection and sequencing of methods of therapy. PAGE 21 

2. Detailed treatments within each method of therapy. PAGE 23 

The booklet and answer sheet for this section are self-explanatory. 



Instructions for PAGE 21. 

The opposite page offers a list of three methods of therapy in every 
possible combination and sequence. Please make ONE selection and record it in 
Column 6 of the answer sheet by filling in the blank next to the appropriate code 
number. 



i 






i 



f er|c 



*\ 

j 



0120 



t 
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► 




n 

U 




SELECTION AND SEQUENCING OF METHODS OF THERAPY 
COLUMN 6 



TREATMENT 



Make one choice in COLUMN 6, 



Hormonal therapy only 
Radiation therapy only 

Surgical therapy onl y “ 

Hormonal therapy followed by rad la t ion Therapy 
Hormonal therapy follov^d by surgical therapy ~ 
Radiation therapy followed by hormonal therapy 
Radiation therapy followed by surgical therapy*" 
Surgical therapy followed by hormonal therapy “ 
Surgical therapy followed by radiation thcrap“ 
Hormonal, then radiation, then surgical therapy"" 
Hormonal, then aurgical, then radiation therapy 
Radiation, then hormonal, then aurgical therapy* 
Radiation, then surgical, then hormonal therapy* 
Surgical, then hormonal, then radiation therapy 
Surgical, then radiation, then hormonal therapy* 
None of those ll 3 ted 



650 

>51 

652 

,6 53 

654 

655 

656 

657 

I 658 

659 

660 

661 

_ 662 

663 

664 

665 



D 

D 



Mark your ONE choice, then proceed to PAGE 22. 



# 




0 

D 

0 

D 





y 






gna 
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INSTRUCTIONS FOR PAGE 23 



s 

9 



i 



Prerequisite # 



Use the opposite page only after you have selected your sequence of treatment 
on PAGE 21 and recorded your choice in Column 6 of the answer sheet. 



On the opposite page, select AS MANY item 3 as you wish, but keep in mind that 
your choice must be related to the sequence of treatment you have previously 
chosen. Mark all of your choices on the answer sheet in Column 7 . 



0 



t 



Note: Columns 8 and 9 on the answer sheet are not used in this test. 



f 



3 



It 

I 

1 

t 



Q 
















i 




D 

0 

0 

D 

0 

0 

0 

Q 

Q 

0 



Q 



9/28/65 



PAGE 23 



DETAILED TREATMENTS 
COLUMN 7 



HORMONAL TREATMENT 



(specify therapy) 



Androgen therap y 

Estrogen therapy 

Progestational therapy" 



Continuous estrogen-progestin therapy_ 
Cyclic estrogen-progestin therapy 



RADIATION TREATMENT 
A. External (specify source) 

(specify targets) 



Conventional x-ray_ 



Supervoltage or telecobalt 

Cancericidal dosage (>5,Q00r): lung fields.. 



Castrating dosage «2,500r): 



upper abdomen^ 
entire abdomsn^ 

pelvic cavity 

ovaries 



B. Internal (specify source 
(radium) 

(specify dosages) 



Vaginal ovoids_ 

Uterine tandem 

Heyman's capsules^ 



SURGICAL TREATMENT 



(specify procedures) 



Dose at vaginal mucosa 
(from vaginal ov~ids) 

Dose at utetine surface 
(from heyman^ capsules 
or tandem) 

Dose at Point A 
(from tandem, ovoids, and 
external sources) 

Dose at Point B 
(from all sources) 



2.000- 3000 r _ 

5.000- 6000 r~ 

10,000-12,000 r 

2.000- 3,000 r_^ 

5.000- 6000 r 

10,000-12,000 r_ 

2.000- 5,000 r 

7,500 r 

15.000 r 

5,000 r ~ 

10.000 r 



Exenteration of pelvis, anterior 
Exenteration of pelvis, 
Exenteration of pelvis, 

Radical hysterectomy 



posterior^ 
total 



Subtotal hysterectom y 

Total hysterectomy^ 

Omentectom y ~ 

Pelvic lymph node dissect! on. 



Sa lp ingo-oophorec tomy , b i 1 at era 1^ 
Salpingo-oophorectomy, unilateral 



When you hive finished this and all preceding pag^s, you have completed this tasfc# 
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750 

“751 

752 

753 
“754 



760 

761 

“762 

763 

"764 

*765 

*766 



770 

771 

772 
*773 
*774 
“775 
“776 
“777 
J78 
-779 
_780 

781 

*782 

*783 



785 

*786 

'787 

*788 

*789 

*790 

*791 

*792 

*793 

*794 
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MEMORANDA ON COURSE CONTENT COMPARABILITY, USE OF VISUAL AIDS, 
AND TIME TO CRITERION RECORDS; SAMPLE TIME SHEET 
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